[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:58:21 dist-test-slave-vnbj krb5kdc[21413](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:58:21 dist-test-slave-vnbj krb5kdc[21413](info): set up 2 sockets
Jun 19 16:58:21 dist-test-slave-vnbj krb5kdc[21413](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260619 16:58:22.435740 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 16:58:22 dist-test-slave-vnbj krb5kdc[21413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:58:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:58:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.231.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
I20260619 16:58:22.562984 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:44177
--webserver_interface=127.20.231.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:44295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:44177
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:22.644223 21426 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:22.644502 21426 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:22.644541 21426 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:22.645704 21426 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:22.645781 21426 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:22.645804 21426 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:22.645828 21426 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:22.645854 21426 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:58:22.647383 21426 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:44295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:44177
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:44177
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21426
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:22.647676 21426 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:22.647908 21426 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 16:58:22.653447 21426 server_base.cc:1034] running on GCE node
W20260619 16:58:22.653445 21431 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:22.653429 21434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:22.653411 21432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:22.654018 21426 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:22.654346 21426 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:22.655524 21426 hybrid_clock.cc:648] HybridClock initialized: now 1781888302655512 us; error 34 us; skew 500 ppm
Jun 19 16:58:22 dist-test-slave-vnbj krb5kdc[21413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888302, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:22.658219 21426 init.cc:377] Logged in from keytab as kudu/127.20.231.190@KRBTEST.COM (short username kudu)
I20260619 16:58:22.658677 21426 webserver.cc:460] Webserver started at http://127.20.231.190:41405/ using document root <none> and password file <none>
I20260619 16:58:22.658972 21426 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:22.659019 21426 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:22.659175 21426 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:22.660260 21426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "f064e9f3bc1145cd80bbffac87a4a49b"
format_stamp: "Formatted at 2026-06-19 16:58:22 on dist-test-slave-vnbj"
server_key: "b7bc8d081410aa2b6c2c7efa377c7ac9"
server_key_iv: "52f404e13b1cbe819ed9b5b52ecff302"
server_key_version: "encryptionkey@0"
I20260619 16:58:22.660655 21426 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "f064e9f3bc1145cd80bbffac87a4a49b"
format_stamp: "Formatted at 2026-06-19 16:58:22 on dist-test-slave-vnbj"
server_key: "b7bc8d081410aa2b6c2c7efa377c7ac9"
server_key_iv: "52f404e13b1cbe819ed9b5b52ecff302"
server_key_version: "encryptionkey@0"
I20260619 16:58:22.662025 21426 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:58:22.662828 21441 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:22.663038 21426 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:58:22.663125 21426 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "f064e9f3bc1145cd80bbffac87a4a49b"
format_stamp: "Formatted at 2026-06-19 16:58:22 on dist-test-slave-vnbj"
server_key: "b7bc8d081410aa2b6c2c7efa377c7ac9"
server_key_iv: "52f404e13b1cbe819ed9b5b52ecff302"
server_key_version: "encryptionkey@0"
I20260619 16:58:22.663180 21426 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:22.674372 21426 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:22.676689 21426 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:22.676803 21426 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:22.681126 21426 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:44177
I20260619 16:58:22.681193 21493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:44177 every 8 connection(s)
I20260619 16:58:22.681552 21426 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 16:58:22.682499 21494 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:22.684878 21494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f064e9f3bc1145cd80bbffac87a4a49b: Bootstrap starting.
I20260619 16:58:22.685491 21494 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f064e9f3bc1145cd80bbffac87a4a49b: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:22.685782 21494 log.cc:824] T 00000000000000000000000000000000 P f064e9f3bc1145cd80bbffac87a4a49b: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:22.686512 21494 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f064e9f3bc1145cd80bbffac87a4a49b: No bootstrap required, opened a new log
I20260619 16:58:22.688426 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 21426
I20260619 16:58:22.688503 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 16:58:22.688755 21494 raft_consensus.cc:348] T 00000000000000000000000000000000 P f064e9f3bc1145cd80bbffac87a4a49b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f064e9f3bc1145cd80bbffac87a4a49b" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 44177 } }
I20260619 16:58:22.688922 21494 raft_consensus.cc:374] T 00000000000000000000000000000000 P f064e9f3bc1145cd80bbffac87a4a49b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:22.688973 21494 raft_consensus.cc:729] T 00000000000000000000000000000000 P f064e9f3bc1145cd80bbffac87a4a49b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f064e9f3bc1145cd80bbffac87a4a49b, State: Initialized, Role: FOLLOWER
I20260619 16:58:22.689159 21494 consensus_queue.cc:260] T 00000000000000000000000000000000 P f064e9f3bc1145cd80bbffac87a4a49b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f064e9f3bc1145cd80bbffac87a4a49b" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 44177 } }
I20260619 16:58:22.689249 21494 raft_consensus.cc:388] T 00000000000000000000000000000000 P f064e9f3bc1145cd80bbffac87a4a49b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:58:22.689232 21406 external_mini_cluster.cc:1428] Setting key 9d96a7223e3a8001460654d01d5650e3
I20260619 16:58:22.689297 21494 raft_consensus.cc:482] T 00000000000000000000000000000000 P f064e9f3bc1145cd80bbffac87a4a49b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:58:22.689364 21494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f064e9f3bc1145cd80bbffac87a4a49b [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:22.690040 21494 raft_consensus.cc:504] T 00000000000000000000000000000000 P f064e9f3bc1145cd80bbffac87a4a49b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f064e9f3bc1145cd80bbffac87a4a49b" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 44177 } }
I20260619 16:58:22.690173 21494 leader_election.cc:302] T 00000000000000000000000000000000 P f064e9f3bc1145cd80bbffac87a4a49b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f064e9f3bc1145cd80bbffac87a4a49b; no voters:
I20260619 16:58:22.690354 21494 leader_election.cc:288] T 00000000000000000000000000000000 P f064e9f3bc1145cd80bbffac87a4a49b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:58:22.690385 21498 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f064e9f3bc1145cd80bbffac87a4a49b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:22.690619 21498 raft_consensus.cc:686] T 00000000000000000000000000000000 P f064e9f3bc1145cd80bbffac87a4a49b [term 1 LEADER]: Becoming Leader. State: Replica: f064e9f3bc1145cd80bbffac87a4a49b, State: Running, Role: LEADER
I20260619 16:58:22.690717 21494 sys_catalog.cc:564] T 00000000000000000000000000000000 P f064e9f3bc1145cd80bbffac87a4a49b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:22.690739 21498 consensus_queue.cc:237] T 00000000000000000000000000000000 P f064e9f3bc1145cd80bbffac87a4a49b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f064e9f3bc1145cd80bbffac87a4a49b" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 44177 } }
I20260619 16:58:22.691339 21500 sys_catalog.cc:455] T 00000000000000000000000000000000 P f064e9f3bc1145cd80bbffac87a4a49b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f064e9f3bc1145cd80bbffac87a4a49b. Latest consensus state: current_term: 1 leader_uuid: "f064e9f3bc1145cd80bbffac87a4a49b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f064e9f3bc1145cd80bbffac87a4a49b" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 44177 } } }
I20260619 16:58:22.691437 21500 sys_catalog.cc:458] T 00000000000000000000000000000000 P f064e9f3bc1145cd80bbffac87a4a49b [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:22.691452 21501 sys_catalog.cc:455] T 00000000000000000000000000000000 P f064e9f3bc1145cd80bbffac87a4a49b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f064e9f3bc1145cd80bbffac87a4a49b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f064e9f3bc1145cd80bbffac87a4a49b" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 44177 } } }
I20260619 16:58:22.691526 21501 sys_catalog.cc:458] T 00000000000000000000000000000000 P f064e9f3bc1145cd80bbffac87a4a49b [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:22.691787 21506 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:58:22.692384 21506 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:58:22.694131 21506 catalog_manager.cc:1347] Generated new cluster ID: f05e0892520c42b3b6b39ad3023fd294
I20260619 16:58:22.694190 21506 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 16:58:22 dist-test-slave-vnbj krb5kdc[21413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888302, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:22.704519 21497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:22.690153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56292 (local address 127.20.231.190:44177)
0619 16:58:22.690571 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:22.690592 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:22.690632 (+ 40us) server_negotiation.cc:407] Connection header received
0619 16:58:22.693168 (+ 2536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:22.693176 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:22.693241 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:22.693401 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:22.695115 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:22.696013 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:22.697043 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:22.697473 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:22.697611 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:22.700964 (+ 3353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:22.700968 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:22.700968 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:22.700978 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:22.702622 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:22.703123 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:22.703130 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:22.703130 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:22.703224 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:22.703502 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:22.703503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:22.703503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:22.703728 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:22.703905 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:22.704163 (+ 258us) server_negotiation.cc:299] Negotiation successful
0619 16:58:22.704307 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":129,"threads_started":1}
I20260619 16:58:22.705123 21506 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:58:22.705770 21506 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:58:22.714984 21506 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f064e9f3bc1145cd80bbffac87a4a49b: Generated new TSK 0
I20260619 16:58:22.715184 21506 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.231.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab.
Entry for principal kudu/127.20.231.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab.
I20260619 16:58:22.746202 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.129:0
--local_ip_for_outbound_sockets=127.20.231.129
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:44177
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:44295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:22.834019 21520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:22.834256 21520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:22.834283 21520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:22.835651 21520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:22.835705 21520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:22.835736 21520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.129
I20260619 16:58:22.837513 21520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:44295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:44177
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.129
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:22.837848 21520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:22.838114 21520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:22.844543 21528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:22.844739 21520 server_base.cc:1034] running on GCE node
W20260619 16:58:22.844532 21526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:22.844515 21525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:22.845088 21520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:22.845410 21520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:22.846608 21520 hybrid_clock.cc:648] HybridClock initialized: now 1781888302846531 us; error 87 us; skew 500 ppm
Jun 19 16:58:22 dist-test-slave-vnbj krb5kdc[21413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888302, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:22.849547 21520 init.cc:377] Logged in from keytab as kudu/127.20.231.129@KRBTEST.COM (short username kudu)
I20260619 16:58:22.850062 21520 webserver.cc:460] Webserver started at http://127.20.231.129:46305/ using document root <none> and password file <none>
I20260619 16:58:22.850337 21520 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:22.850414 21520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:22.850579 21520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:22.851819 21520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-0/data/instance:
uuid: "cca7f2b67df84f1289066dc7bfaf95ea"
format_stamp: "Formatted at 2026-06-19 16:58:22 on dist-test-slave-vnbj"
server_key: "e247cb52c3ad66977e6c29e672ed2677"
server_key_iv: "49e5892f5e15fe5c2cd6154441482022"
server_key_version: "encryptionkey@0"
I20260619 16:58:22.852233 21520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance:
uuid: "cca7f2b67df84f1289066dc7bfaf95ea"
format_stamp: "Formatted at 2026-06-19 16:58:22 on dist-test-slave-vnbj"
server_key: "e247cb52c3ad66977e6c29e672ed2677"
server_key_iv: "49e5892f5e15fe5c2cd6154441482022"
server_key_version: "encryptionkey@0"
I20260619 16:58:22.853813 21520 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 16:58:22.854760 21535 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:22.854970 21520 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:22.855077 21520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-0/wal
uuid: "cca7f2b67df84f1289066dc7bfaf95ea"
format_stamp: "Formatted at 2026-06-19 16:58:22 on dist-test-slave-vnbj"
server_key: "e247cb52c3ad66977e6c29e672ed2677"
server_key_iv: "49e5892f5e15fe5c2cd6154441482022"
server_key_version: "encryptionkey@0"
I20260619 16:58:22.855168 21520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:22.868521 21520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:22.871028 21520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:22.871215 21520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:22.871479 21520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:58:22.871824 21520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:22.871886 21520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:22.871925 21520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:22.871980 21520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:22.878638 21520 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.129:34321
I20260619 16:58:22.878686 21648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.129:34321 every 8 connection(s)
I20260619 16:58:22.879092 21520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
I20260619 16:58:22.881289 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 21520
I20260619 16:58:22.881394 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance
I20260619 16:58:22.881541 21406 external_mini_cluster.cc:1428] Setting key c86de178e9874cbd544603cc58c70c5d
Jun 19 16:58:22 dist-test-slave-vnbj krb5kdc[21413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888302, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:22.890551 21497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:22.879893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:50739 (local address 127.20.231.190:44177)
0619 16:58:22.879966 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:22.879968 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:22.880202 (+ 234us) server_negotiation.cc:407] Connection header received
0619 16:58:22.880696 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:22.880697 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:22.880714 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:22.880782 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:22.882019 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:22.882689 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:22.884380 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:22.884742 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:22.884828 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:22.887412 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:22.887413 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:22.887413 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:22.887420 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:22.888906 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:22.889458 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:22.889459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:22.889460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:22.889522 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:22.889843 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:22.889844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:22.889844 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:22.889965 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:22.890022 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:22.890392 (+ 370us) server_negotiation.cc:299] Negotiation successful
0619 16:58:22.890478 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 16:58:22.890816 21651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:22.879808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:44177 (local address 127.20.231.129:50739)
0619 16:58:22.880031 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:22.880078 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:22.880597 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:22.880868 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:22.880872 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:22.881598 (+ 726us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:22.881915 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:22.881921 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:22.882925 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:22.882927 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:22.884290 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:22.884293 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:22.884854 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:22.884855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:22.885010 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:22.885562 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:22.885572 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:22.887311 (+ 1739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:22.888998 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:22.889005 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:22.889008 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:22.889355 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:22.889604 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:22.889605 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:22.889606 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:22.889756 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:22.890035 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:22.890042 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:22.890229 (+ 187us) client_negotiation.cc:769] Sending connection context
0619 16:58:22.890360 (+ 131us) client_negotiation.cc:240] Negotiation successful
0619 16:58:22.890546 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":98,"threads_started":1}
I20260619 16:58:22.891378 21649 heartbeater.cc:347] Connected to a master server at 127.20.231.190:44177
I20260619 16:58:22.891501 21649 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.20.231.130@KRBTEST.COM; defaulting to no policy
I20260619 16:58:22.891763 21649 heartbeater.cc:511] Master 127.20.231.190:44177 requested a full tablet report, sending...
I20260619 16:58:22.892355 21458 ts_manager.cc:194] Registered new tserver with Master: cca7f2b67df84f1289066dc7bfaf95ea (127.20.231.129:34321)
I20260619 16:58:22.893250 21458 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.129@KRBTEST.COM'} at 127.20.231.129:50739
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab.
Entry for principal kudu/127.20.231.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab.
I20260619 16:58:22.910542 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.130:0
--local_ip_for_outbound_sockets=127.20.231.130
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:44177
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:44295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:22.995795 21654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:22.996007 21654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:22.996033 21654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:22.997175 21654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:22.997217 21654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:22.997251 21654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.130
I20260619 16:58:22.998672 21654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:44295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:44177
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.130
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:22.998929 21654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:22.999130 21654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:23.004221 21662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:23.004374 21654 server_base.cc:1034] running on GCE node
W20260619 16:58:23.004237 21659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:23.004235 21660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:23.004704 21654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:23.004969 21654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:23.006184 21654 hybrid_clock.cc:648] HybridClock initialized: now 1781888303006179 us; error 38 us; skew 500 ppm
Jun 19 16:58:23 dist-test-slave-vnbj krb5kdc[21413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888303, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:23.008823 21654 init.cc:377] Logged in from keytab as kudu/127.20.231.130@KRBTEST.COM (short username kudu)
I20260619 16:58:23.009346 21654 webserver.cc:460] Webserver started at http://127.20.231.130:38571/ using document root <none> and password file <none>
I20260619 16:58:23.009595 21654 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:23.009671 21654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:23.009826 21654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:23.010988 21654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-1/data/instance:
uuid: "0f137b5dade044e2807d6f453b6952ba"
format_stamp: "Formatted at 2026-06-19 16:58:23 on dist-test-slave-vnbj"
server_key: "e7c46134c4300a76027ecef9a7b0a4f5"
server_key_iv: "a8d5db7557d9b166a16a5e8b7c90f4ae"
server_key_version: "encryptionkey@0"
I20260619 16:58:23.011377 21654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance:
uuid: "0f137b5dade044e2807d6f453b6952ba"
format_stamp: "Formatted at 2026-06-19 16:58:23 on dist-test-slave-vnbj"
server_key: "e7c46134c4300a76027ecef9a7b0a4f5"
server_key_iv: "a8d5db7557d9b166a16a5e8b7c90f4ae"
server_key_version: "encryptionkey@0"
I20260619 16:58:23.012804 21654 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:23.013725 21669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:23.013948 21654 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:23.014053 21654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-1/wal
uuid: "0f137b5dade044e2807d6f453b6952ba"
format_stamp: "Formatted at 2026-06-19 16:58:23 on dist-test-slave-vnbj"
server_key: "e7c46134c4300a76027ecef9a7b0a4f5"
server_key_iv: "a8d5db7557d9b166a16a5e8b7c90f4ae"
server_key_version: "encryptionkey@0"
I20260619 16:58:23.014178 21654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:23.031903 21654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:23.034260 21654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:23.034448 21654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:23.034755 21654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:58:23.035187 21654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:23.035247 21654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:23.035291 21654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:23.035319 21654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:23.041659 21654 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.130:46867
I20260619 16:58:23.041704 21782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.130:46867 every 8 connection(s)
I20260619 16:58:23.042063 21654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
I20260619 16:58:23.046104 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 21654
I20260619 16:58:23.046241 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance
I20260619 16:58:23.046434 21406 external_mini_cluster.cc:1428] Setting key cdee4b1eee1a205c2854e4d38d9a8edf
Jun 19 16:58:23 dist-test-slave-vnbj krb5kdc[21413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888303, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:23.053776 21497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:23.043027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:59673 (local address 127.20.231.190:44177)
0619 16:58:23.043089 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:23.043092 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:23.043346 (+ 254us) server_negotiation.cc:407] Connection header received
0619 16:58:23.043907 (+ 561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:23.043909 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:23.043920 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:23.043977 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:23.044959 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:23.045478 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:23.046555 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:23.046826 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:23.046929 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:23.050168 (+ 3239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:23.050170 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:23.050170 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:23.050179 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:23.052281 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:23.052727 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:23.052728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:23.052728 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:23.052795 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:23.053050 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:23.053051 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:23.053051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:23.053227 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:23.053303 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:23.053569 (+ 266us) server_negotiation.cc:299] Negotiation successful
0619 16:58:23.053661 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 16:58:23.053910 21785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:23.043033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:44177 (local address 127.20.231.130:59673)
0619 16:58:23.043218 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:23.043255 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:23.043795 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:23.044057 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:23.044061 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:23.044649 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:23.044847 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:23.044852 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:23.045571 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:23.045572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:23.046452 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:23.046454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:23.046953 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:23.046955 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:23.047092 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:23.047901 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:23.047913 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:23.050037 (+ 2124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:23.052382 (+ 2345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:23.052390 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:23.052392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:23.052645 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:23.052871 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:23.052872 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:23.052872 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:23.052981 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:23.053300 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:23.053305 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:23.053436 (+ 131us) client_negotiation.cc:769] Sending connection context
0619 16:58:23.053528 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 16:58:23.053663 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":99,"threads_started":1}
I20260619 16:58:23.054472 21783 heartbeater.cc:347] Connected to a master server at 127.20.231.190:44177
I20260619 16:58:23.054598 21783 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:23.054793 21783 heartbeater.cc:511] Master 127.20.231.190:44177 requested a full tablet report, sending...
I20260619 16:58:23.055317 21458 ts_manager.cc:194] Registered new tserver with Master: 0f137b5dade044e2807d6f453b6952ba (127.20.231.130:46867)
I20260619 16:58:23.055859 21458 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.130@KRBTEST.COM'} at 127.20.231.130:59673
WARNING: no policy specified for kudu/127.20.231.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab.
Entry for principal kudu/127.20.231.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab.
I20260619 16:58:23.077855 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.131:0
--local_ip_for_outbound_sockets=127.20.231.131
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:44177
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:44295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:23.161767 21788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:23.162034 21788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:23.162081 21788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:23.163461 21788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:23.163537 21788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:23.163591 21788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.131
I20260619 16:58:23.165669 21788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:44295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:44177
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21788
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.131
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:23.166010 21788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:23.166268 21788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:23.171927 21793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:23.171939 21796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:23.171940 21794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:23.172299 21788 server_base.cc:1034] running on GCE node
I20260619 16:58:23.172587 21788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:23.172932 21788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:23.174134 21788 hybrid_clock.cc:648] HybridClock initialized: now 1781888303174131 us; error 49 us; skew 500 ppm
Jun 19 16:58:23 dist-test-slave-vnbj krb5kdc[21413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888303, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:23.177034 21788 init.cc:377] Logged in from keytab as kudu/127.20.231.131@KRBTEST.COM (short username kudu)
I20260619 16:58:23.177503 21788 webserver.cc:460] Webserver started at http://127.20.231.131:42655/ using document root <none> and password file <none>
I20260619 16:58:23.177752 21788 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:23.177821 21788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:23.177979 21788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:23.179265 21788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-2/data/instance:
uuid: "e38ce794f8704dd3a86de971ff540e85"
format_stamp: "Formatted at 2026-06-19 16:58:23 on dist-test-slave-vnbj"
server_key: "60b8ba2f93ab907514e7658370fd07ac"
server_key_iv: "86d9adae38cb18b8edf1893e2ee10cc5"
server_key_version: "encryptionkey@0"
I20260619 16:58:23.179642 21788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance:
uuid: "e38ce794f8704dd3a86de971ff540e85"
format_stamp: "Formatted at 2026-06-19 16:58:23 on dist-test-slave-vnbj"
server_key: "60b8ba2f93ab907514e7658370fd07ac"
server_key_iv: "86d9adae38cb18b8edf1893e2ee10cc5"
server_key_version: "encryptionkey@0"
I20260619 16:58:23.181063 21788 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:23.181915 21803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:23.182114 21788 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:23.182219 21788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-2/wal
uuid: "e38ce794f8704dd3a86de971ff540e85"
format_stamp: "Formatted at 2026-06-19 16:58:23 on dist-test-slave-vnbj"
server_key: "60b8ba2f93ab907514e7658370fd07ac"
server_key_iv: "86d9adae38cb18b8edf1893e2ee10cc5"
server_key_version: "encryptionkey@0"
I20260619 16:58:23.182307 21788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:23.197808 21788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:23.200027 21788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:23.200213 21788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:23.200451 21788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:58:23.200795 21788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:23.200858 21788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:23.200922 21788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:23.200950 21788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:23.207283 21788 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.131:35691
I20260619 16:58:23.207335 21916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.131:35691 every 8 connection(s)
I20260619 16:58:23.207702 21788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
I20260619 16:58:23.213165 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 21788
I20260619 16:58:23.213238 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance
I20260619 16:58:23.213413 21406 external_mini_cluster.cc:1428] Setting key 4a929005b981ba5f3ecd4fa95ad72d86
Jun 19 16:58:23 dist-test-slave-vnbj krb5kdc[21413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888303, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:23.217491 21497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:23.208441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:59997 (local address 127.20.231.190:44177)
0619 16:58:23.208504 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:23.208506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:23.208684 (+ 178us) server_negotiation.cc:407] Connection header received
0619 16:58:23.209275 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:23.209277 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:23.209290 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:23.209369 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:23.210365 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:23.210921 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:23.211938 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:23.212142 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:23.212228 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:23.214731 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:23.214732 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:23.214733 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:23.214742 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:23.216091 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:23.216483 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:23.216484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:23.216484 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:23.216534 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:23.216842 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:23.216843 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:23.216843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:23.216945 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:23.216995 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:23.217363 (+ 368us) server_negotiation.cc:299] Negotiation successful
0619 16:58:23.217425 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 16:58:23.217666 21919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:23.208411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:44177 (local address 127.20.231.131:59997)
0619 16:58:23.208556 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:23.208598 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:23.209155 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:23.209445 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:23.209448 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:23.210035 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:23.210232 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:23.210237 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:23.211014 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:23.211015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:23.211829 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:23.211831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:23.212238 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:23.212239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:23.212375 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:23.212895 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:23.212904 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:23.214623 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:23.216181 (+ 1558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:23.216186 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:23.216189 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:23.216413 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:23.216628 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:23.216629 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:23.216630 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:23.216775 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:23.217010 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:23.217015 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:23.217212 (+ 197us) client_negotiation.cc:769] Sending connection context
0619 16:58:23.217310 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 16:58:23.217447 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":72,"threads_started":1}
I20260619 16:58:23.218132 21917 heartbeater.cc:347] Connected to a master server at 127.20.231.190:44177
I20260619 16:58:23.218245 21917 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:23.218405 21917 heartbeater.cc:511] Master 127.20.231.190:44177 requested a full tablet report, sending...
I20260619 16:58:23.218755 21458 ts_manager.cc:194] Registered new tserver with Master: e38ce794f8704dd3a86de971ff540e85 (127.20.231.131:35691)
I20260619 16:58:23.219152 21458 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.131@KRBTEST.COM'} at 127.20.231.131:59997
I20260619 16:58:23.225699 21406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 16:58:23 dist-test-slave-vnbj krb5kdc[21413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888303, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 16:58:23 dist-test-slave-vnbj krb5kdc[21413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888303, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.231.129@KRBTEST.COM
I20260619 16:58:23.250239 21924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:23.241799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35204 (local address 127.20.231.129:34321)
0619 16:58:23.241976 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:23.241981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:23.241991 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:58:23.242019 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:23.242022 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:23.242037 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:23.242118 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:23.242708 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:23.243523 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:23.244233 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:23.244570 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:23.244639 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:23.246990 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:23.247000 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:23.247000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:23.247007 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:23.248843 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:23.249274 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:23.249277 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:23.249278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:23.249364 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:23.249580 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:23.249580 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:23.249581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:23.249811 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:23.249942 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:23.250065 (+ 123us) server_negotiation.cc:299] Negotiation successful
0619 16:58:23.250148 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":100,"threads_started":1}
W20260619 16:58:23.250703 21563 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35204
Jun 19 16:58:23 dist-test-slave-vnbj krb5kdc[21413](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 16:58:23 dist-test-slave-vnbj krb5kdc[21413](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.129@KRBTEST.COM
I20260619 16:58:23.275408 21924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:23.267254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35212 (local address 127.20.231.129:34321)
0619 16:58:23.267331 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:23.267334 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:23.267444 (+ 110us) server_negotiation.cc:407] Connection header received
0619 16:58:23.267496 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:23.267497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:23.267506 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:23.267571 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:23.268278 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:23.268873 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:23.269743 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:23.269951 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:23.270017 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:23.272740 (+ 2723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:23.272742 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:23.272742 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:23.272751 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:23.274205 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:23.274623 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:23.274623 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:23.274624 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:23.274673 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:23.274960 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:23.274961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:23.274961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:23.275058 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:23.275151 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:23.275263 (+ 112us) server_negotiation.cc:299] Negotiation successful
0619 16:58:23.275338 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 16:58:23.276110 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 21520
I20260619 16:58:23.281689 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 21654
I20260619 16:58:23.286113 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 21788
I20260619 16:58:23.290506 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 21426
2026-06-19T16:58:23Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1953 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:58:23 dist-test-slave-vnbj krb5kdc[21931](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:58:23 dist-test-slave-vnbj krb5kdc[21931](info): set up 2 sockets
Jun 19 16:58:23 dist-test-slave-vnbj krb5kdc[21931](info): commencing operation
krb5kdc: starting...
W20260619 16:58:24.343014 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 16:58:24 dist-test-slave-vnbj krb5kdc[21931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:58:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:58:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.231.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
I20260619 16:58:24.470600 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:34643
--webserver_interface=127.20.231.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:36255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:34643
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:24.551452 21944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:24.551712 21944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:24.551748 21944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:24.552943 21944 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:24.553015 21944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:24.553041 21944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:24.553074 21944 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:24.553098 21944 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:58:24.554646 21944 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:36255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:34643
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:34643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:24.554968 21944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:24.555212 21944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 16:58:24.560290 21944 server_base.cc:1034] running on GCE node
W20260619 16:58:24.560252 21949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:24.560218 21952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:24.560213 21950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:24.560803 21944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:24.561141 21944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:24.562345 21944 hybrid_clock.cc:648] HybridClock initialized: now 1781888304562329 us; error 27 us; skew 500 ppm
Jun 19 16:58:24 dist-test-slave-vnbj krb5kdc[21931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888304, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:24.565173 21944 init.cc:377] Logged in from keytab as kudu/127.20.231.190@KRBTEST.COM (short username kudu)
I20260619 16:58:24.565685 21944 webserver.cc:460] Webserver started at http://127.20.231.190:46345/ using document root <none> and password file <none>
I20260619 16:58:24.565956 21944 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:24.566035 21944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:24.566210 21944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:24.567533 21944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "02f526b97ced44babfdf245e38a98326"
format_stamp: "Formatted at 2026-06-19 16:58:24 on dist-test-slave-vnbj"
server_key: "bea52b98d2669c6f9a61ed5fe2966751"
server_key_iv: "9d235bc733394b92d5302a7c6572a9ec"
server_key_version: "encryptionkey@0"
I20260619 16:58:24.567991 21944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "02f526b97ced44babfdf245e38a98326"
format_stamp: "Formatted at 2026-06-19 16:58:24 on dist-test-slave-vnbj"
server_key: "bea52b98d2669c6f9a61ed5fe2966751"
server_key_iv: "9d235bc733394b92d5302a7c6572a9ec"
server_key_version: "encryptionkey@0"
I20260619 16:58:24.569497 21944 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 16:58:24.570475 21959 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:24.570770 21944 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:24.570950 21944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "02f526b97ced44babfdf245e38a98326"
format_stamp: "Formatted at 2026-06-19 16:58:24 on dist-test-slave-vnbj"
server_key: "bea52b98d2669c6f9a61ed5fe2966751"
server_key_iv: "9d235bc733394b92d5302a7c6572a9ec"
server_key_version: "encryptionkey@0"
I20260619 16:58:24.571038 21944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:24.580372 21944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:24.582392 21944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:24.582561 21944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:24.586690 21944 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:34643
I20260619 16:58:24.586728 22011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:34643 every 8 connection(s)
I20260619 16:58:24.587175 21944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 16:58:24.588135 22012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:24.590605 22012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02f526b97ced44babfdf245e38a98326: Bootstrap starting.
I20260619 16:58:24.591266 22012 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 02f526b97ced44babfdf245e38a98326: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:24.591513 22012 log.cc:824] T 00000000000000000000000000000000 P 02f526b97ced44babfdf245e38a98326: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:24.592257 22012 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 02f526b97ced44babfdf245e38a98326: No bootstrap required, opened a new log
I20260619 16:58:24.593997 22012 raft_consensus.cc:348] T 00000000000000000000000000000000 P 02f526b97ced44babfdf245e38a98326 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02f526b97ced44babfdf245e38a98326" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 34643 } }
I20260619 16:58:24.594154 22012 raft_consensus.cc:374] T 00000000000000000000000000000000 P 02f526b97ced44babfdf245e38a98326 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:24.594182 22012 raft_consensus.cc:729] T 00000000000000000000000000000000 P 02f526b97ced44babfdf245e38a98326 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02f526b97ced44babfdf245e38a98326, State: Initialized, Role: FOLLOWER
I20260619 16:58:24.594348 22012 consensus_queue.cc:260] T 00000000000000000000000000000000 P 02f526b97ced44babfdf245e38a98326 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02f526b97ced44babfdf245e38a98326" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 34643 } }
I20260619 16:58:24.594439 22012 raft_consensus.cc:388] T 00000000000000000000000000000000 P 02f526b97ced44babfdf245e38a98326 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:58:24.594465 22012 raft_consensus.cc:482] T 00000000000000000000000000000000 P 02f526b97ced44babfdf245e38a98326 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:58:24.594506 22012 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 02f526b97ced44babfdf245e38a98326 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:24.595155 22012 raft_consensus.cc:504] T 00000000000000000000000000000000 P 02f526b97ced44babfdf245e38a98326 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02f526b97ced44babfdf245e38a98326" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 34643 } }
I20260619 16:58:24.595249 22012 leader_election.cc:302] T 00000000000000000000000000000000 P 02f526b97ced44babfdf245e38a98326 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 02f526b97ced44babfdf245e38a98326; no voters:
I20260619 16:58:24.595424 22012 leader_election.cc:288] T 00000000000000000000000000000000 P 02f526b97ced44babfdf245e38a98326 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:58:24.595463 22015 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 02f526b97ced44babfdf245e38a98326 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:24.595654 22015 raft_consensus.cc:686] T 00000000000000000000000000000000 P 02f526b97ced44babfdf245e38a98326 [term 1 LEADER]: Becoming Leader. State: Replica: 02f526b97ced44babfdf245e38a98326, State: Running, Role: LEADER
I20260619 16:58:24.595738 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 21944
I20260619 16:58:24.595749 22015 consensus_queue.cc:237] T 00000000000000000000000000000000 P 02f526b97ced44babfdf245e38a98326 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02f526b97ced44babfdf245e38a98326" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 34643 } }
I20260619 16:58:24.595880 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 16:58:24.595759 22012 sys_catalog.cc:564] T 00000000000000000000000000000000 P 02f526b97ced44babfdf245e38a98326 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:24.596076 21406 external_mini_cluster.cc:1428] Setting key 948f01b2f84cb645b04bc775c8bc4d7b
I20260619 16:58:24.596104 22016 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02f526b97ced44babfdf245e38a98326 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 02f526b97ced44babfdf245e38a98326. Latest consensus state: current_term: 1 leader_uuid: "02f526b97ced44babfdf245e38a98326" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02f526b97ced44babfdf245e38a98326" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 34643 } } }
I20260619 16:58:24.596717 22016 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02f526b97ced44babfdf245e38a98326 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:24.597062 22026 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:58:24.597649 22026 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:58:24.597879 22017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 02f526b97ced44babfdf245e38a98326 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "02f526b97ced44babfdf245e38a98326" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "02f526b97ced44babfdf245e38a98326" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 34643 } } }
I20260619 16:58:24.597971 22017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 02f526b97ced44babfdf245e38a98326 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:24.599573 22026 catalog_manager.cc:1347] Generated new cluster ID: 549a9c23e970464fb47c96aa01eb7fd0
I20260619 16:58:24.599633 22026 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 16:58:24 dist-test-slave-vnbj krb5kdc[21931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:24.606379 22026 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:58:24.607178 22026 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:58:24.609344 22020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:24.596622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42948 (local address 127.20.231.190:34643)
0619 16:58:24.597174 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:24.597185 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:24.597221 (+ 36us) server_negotiation.cc:407] Connection header received
0619 16:58:24.597356 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:24.597365 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:24.597431 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:24.598525 (+ 1094us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:24.598663 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:24.600018 (+ 1355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:24.601241 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:24.601867 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:24.601973 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:24.605092 (+ 3119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:24.605097 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:24.605098 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:24.605113 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:24.607790 (+ 2677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:24.608184 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:24.608189 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:24.608190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:24.608283 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:24.608491 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:24.608491 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:24.608492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:24.608714 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:24.608895 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:24.608974 (+ 79us) server_negotiation.cc:299] Negotiation successful
0619 16:58:24.609156 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":85,"threads_started":1}
I20260619 16:58:24.610745 22026 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 02f526b97ced44babfdf245e38a98326: Generated new TSK 0
I20260619 16:58:24.610962 22026 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.231.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab.
Entry for principal kudu/127.20.231.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab.
I20260619 16:58:24.642804 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.129:0
--local_ip_for_outbound_sockets=127.20.231.129
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:34643
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:36255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:24.740268 22038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:24.740480 22038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:24.740505 22038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:24.741724 22038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:24.741771 22038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:24.741801 22038 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.129
I20260619 16:58:24.743515 22038 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:36255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:34643
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.129
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:24.743768 22038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:24.744031 22038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:24.749320 22044 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:24.749370 22043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:24.749454 22046 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:24.749696 22038 server_base.cc:1034] running on GCE node
I20260619 16:58:24.749971 22038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:24.750241 22038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:24.751389 22038 hybrid_clock.cc:648] HybridClock initialized: now 1781888304751367 us; error 39 us; skew 500 ppm
Jun 19 16:58:24 dist-test-slave-vnbj krb5kdc[21931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888304, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:24.753942 22038 init.cc:377] Logged in from keytab as kudu/127.20.231.129@KRBTEST.COM (short username kudu)
I20260619 16:58:24.754446 22038 webserver.cc:460] Webserver started at http://127.20.231.129:41113/ using document root <none> and password file <none>
I20260619 16:58:24.754683 22038 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:24.754736 22038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:24.754922 22038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:24.756016 22038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-0/data/instance:
uuid: "e54366b1d44e45ed907a5936a1ac6c65"
format_stamp: "Formatted at 2026-06-19 16:58:24 on dist-test-slave-vnbj"
server_key: "1f1b195a6c9be48d62a70b41ceb87c8d"
server_key_iv: "e8ced189074571f90cbc84b396b11e8e"
server_key_version: "encryptionkey@0"
I20260619 16:58:24.756387 22038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance:
uuid: "e54366b1d44e45ed907a5936a1ac6c65"
format_stamp: "Formatted at 2026-06-19 16:58:24 on dist-test-slave-vnbj"
server_key: "1f1b195a6c9be48d62a70b41ceb87c8d"
server_key_iv: "e8ced189074571f90cbc84b396b11e8e"
server_key_version: "encryptionkey@0"
I20260619 16:58:24.757792 22038 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:58:24.758564 22053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:24.758827 22038 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:24.758940 22038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-0/wal
uuid: "e54366b1d44e45ed907a5936a1ac6c65"
format_stamp: "Formatted at 2026-06-19 16:58:24 on dist-test-slave-vnbj"
server_key: "1f1b195a6c9be48d62a70b41ceb87c8d"
server_key_iv: "e8ced189074571f90cbc84b396b11e8e"
server_key_version: "encryptionkey@0"
I20260619 16:58:24.759020 22038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:24.780880 22038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:24.783135 22038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:24.783259 22038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:24.783468 22038 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:58:24.783768 22038 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:24.783800 22038 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:24.783824 22038 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:24.783836 22038 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:24.789812 22038 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.129:33955
I20260619 16:58:24.789921 22166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.129:33955 every 8 connection(s)
I20260619 16:58:24.790266 22038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
Jun 19 16:58:24 dist-test-slave-vnbj krb5kdc[21931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888304, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:24.799028 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 22038
I20260619 16:58:24.799113 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance
I20260619 16:58:24.799245 21406 external_mini_cluster.cc:1428] Setting key 3531337046b1cea7488d216be49256a7
I20260619 16:58:24.801589 22020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:24.791098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:46557 (local address 127.20.231.190:34643)
0619 16:58:24.791206 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:24.791208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:24.791405 (+ 197us) server_negotiation.cc:407] Connection header received
0619 16:58:24.792390 (+ 985us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:24.792391 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:24.792403 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:24.792491 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:24.793151 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:24.793665 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:24.794777 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:24.795026 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:24.795096 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:24.797827 (+ 2731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:24.797829 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:24.797829 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:24.797844 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:24.800023 (+ 2179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:24.800488 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:24.800489 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:24.800490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:24.800549 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:24.800872 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:24.800873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:24.800873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:24.800996 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:24.801079 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:24.801400 (+ 321us) server_negotiation.cc:299] Negotiation successful
0619 16:58:24.801490 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260619 16:58:24.801801 22169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:24.791081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:34643 (local address 127.20.231.129:46557)
0619 16:58:24.791254 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:24.791298 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:24.792255 (+ 957us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:24.792571 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:24.792575 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:24.792842 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:24.793045 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:24.793048 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:24.793824 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:24.793825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:24.794666 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:24.794669 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:24.795146 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:24.795148 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:24.795268 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:24.795984 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:24.795994 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:24.797726 (+ 1732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:24.800127 (+ 2401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:24.800135 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:24.800137 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:24.800387 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:24.800639 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:24.800640 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:24.800640 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:24.800793 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:24.801068 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:24.801075 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:24.801246 (+ 171us) client_negotiation.cc:769] Sending connection context
0619 16:58:24.801362 (+ 116us) client_negotiation.cc:240] Negotiation successful
0619 16:58:24.801538 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":86,"threads_started":1}
I20260619 16:58:24.802332 22167 heartbeater.cc:347] Connected to a master server at 127.20.231.190:34643
I20260619 16:58:24.802436 22167 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:24.802634 22167 heartbeater.cc:511] Master 127.20.231.190:34643 requested a full tablet report, sending...
I20260619 16:58:24.803289 21976 ts_manager.cc:194] Registered new tserver with Master: e54366b1d44e45ed907a5936a1ac6c65 (127.20.231.129:33955)
I20260619 16:58:24.804134 21976 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.129@KRBTEST.COM'} at 127.20.231.129:46557
WARNING: no policy specified for kudu/127.20.231.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab.
Entry for principal kudu/127.20.231.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab.
I20260619 16:58:24.827111 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.130:0
--local_ip_for_outbound_sockets=127.20.231.130
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:34643
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:36255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:24.911566 22172 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:24.911902 22172 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:24.911944 22172 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:24.913189 22172 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:24.913252 22172 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:24.913296 22172 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.130
I20260619 16:58:24.914812 22172 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:36255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:34643
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22172
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.130
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:24.915113 22172 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:24.915371 22172 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:24.920611 22178 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:24.920603 22180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:24.920603 22177 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:24.920903 22172 server_base.cc:1034] running on GCE node
I20260619 16:58:24.921146 22172 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:24.921478 22172 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:24.922679 22172 hybrid_clock.cc:648] HybridClock initialized: now 1781888304922626 us; error 64 us; skew 500 ppm
Jun 19 16:58:24 dist-test-slave-vnbj krb5kdc[21931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888304, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:24.925413 22172 init.cc:377] Logged in from keytab as kudu/127.20.231.130@KRBTEST.COM (short username kudu)
I20260619 16:58:24.925858 22172 webserver.cc:460] Webserver started at http://127.20.231.130:45617/ using document root <none> and password file <none>
I20260619 16:58:24.926092 22172 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:24.926158 22172 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:24.926309 22172 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:24.927459 22172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-1/data/instance:
uuid: "21fd7d5cb70a4eff9d89bceacd0a4267"
format_stamp: "Formatted at 2026-06-19 16:58:24 on dist-test-slave-vnbj"
server_key: "5aa892960dcf60bf546c60d910138722"
server_key_iv: "6055537f051411dbf56e2ae9a277b161"
server_key_version: "encryptionkey@0"
I20260619 16:58:24.927843 22172 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance:
uuid: "21fd7d5cb70a4eff9d89bceacd0a4267"
format_stamp: "Formatted at 2026-06-19 16:58:24 on dist-test-slave-vnbj"
server_key: "5aa892960dcf60bf546c60d910138722"
server_key_iv: "6055537f051411dbf56e2ae9a277b161"
server_key_version: "encryptionkey@0"
I20260619 16:58:24.929176 22172 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:58:24.930004 22187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:24.930243 22172 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:24.930344 22172 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-1/wal
uuid: "21fd7d5cb70a4eff9d89bceacd0a4267"
format_stamp: "Formatted at 2026-06-19 16:58:24 on dist-test-slave-vnbj"
server_key: "5aa892960dcf60bf546c60d910138722"
server_key_iv: "6055537f051411dbf56e2ae9a277b161"
server_key_version: "encryptionkey@0"
I20260619 16:58:24.930426 22172 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:24.964584 22172 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:24.967149 22172 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:24.967320 22172 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:24.967568 22172 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:58:24.967900 22172 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:24.967957 22172 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:24.967999 22172 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:24.968034 22172 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:24.974705 22172 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.130:42927
I20260619 16:58:24.974787 22300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.130:42927 every 8 connection(s)
I20260619 16:58:24.975145 22172 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
Jun 19 16:58:24 dist-test-slave-vnbj krb5kdc[21931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888304, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:24.982419 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 22172
I20260619 16:58:24.982513 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance
I20260619 16:58:24.982692 21406 external_mini_cluster.cc:1428] Setting key 7082b8bc27e54a957e464af33a39ad08
I20260619 16:58:24.985550 22020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:24.975924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:52453 (local address 127.20.231.190:34643)
0619 16:58:24.975973 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:24.975975 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:24.976213 (+ 238us) server_negotiation.cc:407] Connection header received
0619 16:58:24.977162 (+ 949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:24.977163 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:24.977174 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:24.977235 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:24.977823 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:24.978319 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:24.979380 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:24.979566 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:24.979623 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:24.982281 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:24.982283 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:24.982283 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:24.982292 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:24.984001 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:24.984445 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:24.984446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:24.984446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:24.984509 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:24.984769 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:24.984770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:24.984770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:24.984920 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:24.984997 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:24.985415 (+ 418us) server_negotiation.cc:299] Negotiation successful
0619 16:58:24.985485 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 16:58:24.985884 22303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:24.975899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:34643 (local address 127.20.231.130:52453)
0619 16:58:24.976058 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:24.976101 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:24.977062 (+ 961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:24.977290 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:24.977293 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:24.977549 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:24.977750 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:24.977754 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:24.978403 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:24.978403 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:24.979305 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:24.979308 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:24.979720 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:24.979721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:24.979849 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:24.980367 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:24.980376 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:24.982151 (+ 1775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:24.984093 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:24.984099 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:24.984101 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:24.984365 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:24.984581 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:24.984581 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:24.984582 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:24.984693 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:24.985003 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:24.985010 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:24.985257 (+ 247us) client_negotiation.cc:769] Sending connection context
0619 16:58:24.985394 (+ 137us) client_negotiation.cc:240] Negotiation successful
0619 16:58:24.985597 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":90,"threads_started":1}
I20260619 16:58:24.986493 22301 heartbeater.cc:347] Connected to a master server at 127.20.231.190:34643
I20260619 16:58:24.986596 22301 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:24.986817 22301 heartbeater.cc:511] Master 127.20.231.190:34643 requested a full tablet report, sending...
I20260619 16:58:24.987267 21976 ts_manager.cc:194] Registered new tserver with Master: 21fd7d5cb70a4eff9d89bceacd0a4267 (127.20.231.130:42927)
I20260619 16:58:24.987694 21976 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.130@KRBTEST.COM'} at 127.20.231.130:52453
WARNING: no policy specified for kudu/127.20.231.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab.
Entry for principal kudu/127.20.231.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab.
I20260619 16:58:25.010087 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.131:0
--local_ip_for_outbound_sockets=127.20.231.131
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:34643
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:36255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:25.092134 22306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:25.092363 22306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:25.092388 22306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:25.093580 22306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:25.093621 22306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:25.093647 22306 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.131
I20260619 16:58:25.095229 22306 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:36255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:34643
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.131
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:25.095465 22306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:25.095754 22306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:25.101776 22311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:25.101862 22306 server_base.cc:1034] running on GCE node
W20260619 16:58:25.101913 22312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:25.101785 22314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:25.102218 22306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:25.102465 22306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:25.103720 22306 hybrid_clock.cc:648] HybridClock initialized: now 1781888305103691 us; error 59 us; skew 500 ppm
Jun 19 16:58:25 dist-test-slave-vnbj krb5kdc[21931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888305, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:25.106608 22306 init.cc:377] Logged in from keytab as kudu/127.20.231.131@KRBTEST.COM (short username kudu)
I20260619 16:58:25.107179 22306 webserver.cc:460] Webserver started at http://127.20.231.131:33117/ using document root <none> and password file <none>
I20260619 16:58:25.107424 22306 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:25.107501 22306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:25.107668 22306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:25.108834 22306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-2/data/instance:
uuid: "d8e1d70b01aa437d9d3faf6135784a6d"
format_stamp: "Formatted at 2026-06-19 16:58:25 on dist-test-slave-vnbj"
server_key: "dee57f89d7c0b07aec6a9e99dd132615"
server_key_iv: "0ab9a37465a48efd4137437abb23be23"
server_key_version: "encryptionkey@0"
I20260619 16:58:25.109238 22306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance:
uuid: "d8e1d70b01aa437d9d3faf6135784a6d"
format_stamp: "Formatted at 2026-06-19 16:58:25 on dist-test-slave-vnbj"
server_key: "dee57f89d7c0b07aec6a9e99dd132615"
server_key_iv: "0ab9a37465a48efd4137437abb23be23"
server_key_version: "encryptionkey@0"
I20260619 16:58:25.110814 22306 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:58:25.111814 22321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:25.112006 22306 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:25.112087 22306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-2/wal
uuid: "d8e1d70b01aa437d9d3faf6135784a6d"
format_stamp: "Formatted at 2026-06-19 16:58:25 on dist-test-slave-vnbj"
server_key: "dee57f89d7c0b07aec6a9e99dd132615"
server_key_iv: "0ab9a37465a48efd4137437abb23be23"
server_key_version: "encryptionkey@0"
I20260619 16:58:25.112140 22306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:25.138087 22306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:25.140657 22306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:25.140784 22306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:25.140987 22306 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:58:25.141327 22306 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:25.141364 22306 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:25.141387 22306 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:25.141433 22306 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:25.147553 22306 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.131:37653
I20260619 16:58:25.147617 22434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.131:37653 every 8 connection(s)
I20260619 16:58:25.147956 22306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
Jun 19 16:58:25 dist-test-slave-vnbj krb5kdc[21931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888305, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:25.155898 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 22306
I20260619 16:58:25.156015 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance
I20260619 16:58:25.156214 21406 external_mini_cluster.cc:1428] Setting key f4cf55a3fdea9a50c640b4b3f7390c3f
I20260619 16:58:25.159017 22020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:25.148769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:35993 (local address 127.20.231.190:34643)
0619 16:58:25.148816 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:25.148818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:25.149094 (+ 276us) server_negotiation.cc:407] Connection header received
0619 16:58:25.150027 (+ 933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:25.150028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:25.150041 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:25.150115 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:25.150973 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:25.151519 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:25.152617 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:25.152900 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:25.152970 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:25.155455 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:25.155457 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:25.155457 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:25.155465 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:25.157616 (+ 2151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:25.158053 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:25.158054 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:25.158055 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:25.158113 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:25.158415 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:25.158415 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:25.158416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:25.158523 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:25.158578 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:25.158825 (+ 247us) server_negotiation.cc:299] Negotiation successful
0619 16:58:25.158954 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 16:58:25.159287 22437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:25.148721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:34643 (local address 127.20.231.131:35993)
0619 16:58:25.148917 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:25.148961 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:25.149897 (+ 936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:25.150262 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:25.150267 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:25.150574 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:25.150791 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:25.150796 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:25.151607 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:25.151608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:25.152521 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:25.152524 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:25.152989 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:25.152990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:25.153101 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:25.153660 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:25.153669 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:25.155361 (+ 1692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:25.157704 (+ 2343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:25.157710 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:25.157713 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:25.157964 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:25.158222 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:25.158223 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:25.158223 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:25.158338 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:25.158583 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:25.158588 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:25.158710 (+ 122us) client_negotiation.cc:769] Sending connection context
0619 16:58:25.158796 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 16:58:25.159005 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":129,"threads_started":1}
I20260619 16:58:25.159802 22435 heartbeater.cc:347] Connected to a master server at 127.20.231.190:34643
I20260619 16:58:25.159917 22435 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:25.160143 22435 heartbeater.cc:511] Master 127.20.231.190:34643 requested a full tablet report, sending...
I20260619 16:58:25.160465 21976 ts_manager.cc:194] Registered new tserver with Master: d8e1d70b01aa437d9d3faf6135784a6d (127.20.231.131:37653)
I20260619 16:58:25.160801 21976 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.131@KRBTEST.COM'} at 127.20.231.131:35993
I20260619 16:58:25.168723 21406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 16:58:25.177460 22020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:25.170345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42964 (local address 127.20.231.190:34643)
0619 16:58:25.170440 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:25.170442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:25.170544 (+ 102us) server_negotiation.cc:407] Connection header received
0619 16:58:25.170646 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:25.170647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:25.170659 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:25.170840 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:25.171505 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:25.172054 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:25.172859 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:25.173078 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:25.173193 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:25.174588 (+ 1395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:25.174590 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:25.174590 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:25.174598 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:25.176096 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:25.176533 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:25.176535 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:25.176535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:25.176598 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:25.176835 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:25.176836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:25.176836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:25.177001 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:25.177117 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:25.177252 (+ 135us) server_negotiation.cc:299] Negotiation successful
0619 16:58:25.177341 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 16:58:25.180121 21976 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42964:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:58:25.181759 21976 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:58:25.189168 22451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:25.185060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41878 (local address 127.20.231.129:33955)
0619 16:58:25.185213 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:25.185218 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:25.185227 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:58:25.185267 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:25.185269 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:25.185285 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:25.185589 (+ 304us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:25.185666 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:25.186700 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:25.187976 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:25.188656 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:25.188739 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:25.188796 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:25.189011 (+ 215us) server_negotiation.cc:299] Negotiation successful
0619 16:58:25.189054 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":82,"threads_started":1}
I20260619 16:58:25.189836 22449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:25.184843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36680 (local address 127.20.231.130:42927)
0619 16:58:25.185048 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:25.185054 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:25.185066 (+ 12us) server_negotiation.cc:407] Connection header received
0619 16:58:25.185112 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:25.185115 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:25.185134 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:25.185274 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:25.185467 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:25.186617 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:25.188336 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:25.189083 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:25.189161 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:25.189215 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:25.189659 (+ 444us) server_negotiation.cc:299] Negotiation successful
0619 16:58:25.189717 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":89,"threads_started":1}
I20260619 16:58:25.190636 22450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:25.184882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47284 (local address 127.20.231.131:37653)
0619 16:58:25.185139 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:25.185189 (+ 50us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:25.185374 (+ 185us) server_negotiation.cc:407] Connection header received
0619 16:58:25.185417 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:25.185420 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:25.185436 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:25.185512 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:25.185744 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:25.186503 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:25.189369 (+ 2866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:25.190276 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:25.190365 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:25.190417 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:25.190514 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 16:58:25.190551 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":132,"threads_started":1}
I20260619 16:58:25.189204 22446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:25.184586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:33955 (local address 127.0.0.1:41878)
0619 16:58:25.185036 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:25.185047 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:25.185085 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:25.185454 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:25.185455 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:25.185457 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:25.185513 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:25.185515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:25.186783 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:25.186784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:25.187886 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:25.187889 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:25.188794 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:25.188795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:25.188884 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:25.188898 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 16:58:25.188983 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 16:58:25.189055 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"mutex_wait_us":38,"thread_start_us":72,"threads_started":1}
I20260619 16:58:25.191072 22447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:25.184586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:42927 (local address 127.0.0.1:36680)
0619 16:58:25.184807 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:25.184852 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:25.184923 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:25.185294 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:25.185298 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:25.185301 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:25.185380 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:25.185382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:25.186722 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:25.186723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:25.188236 (+ 1513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:25.188238 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:25.189189 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:25.189191 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:25.189278 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:25.189285 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 16:58:25.189373 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 16:58:25.189431 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":100,"thread_start_us":79,"threads_started":1}
I20260619 16:58:25.191071 22448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:25.184614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:37653 (local address 127.0.0.1:47284)
0619 16:58:25.185245 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:25.185256 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:25.185316 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:25.185607 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:25.185608 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:25.185610 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:25.185659 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:25.185662 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:25.186621 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:25.186622 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:25.188234 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:25.188238 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:25.190374 (+ 2136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:25.190375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:25.190441 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:25.190445 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:58:25.190501 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 16:58:25.190547 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"mutex_wait_us":80,"thread_start_us":202,"threads_started":1}
I20260619 16:58:25.191808 22235 tablet_service.cc:1467] Processing CreateTablet for tablet 4bac9516c95a460a8168a10d45116e65 (DEFAULT_TABLE table=test-table [id=2c533b96bc3948c59a87fd642df56d25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:25.191808 22101 tablet_service.cc:1467] Processing CreateTablet for tablet 4bac9516c95a460a8168a10d45116e65 (DEFAULT_TABLE table=test-table [id=2c533b96bc3948c59a87fd642df56d25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:25.191908 22369 tablet_service.cc:1467] Processing CreateTablet for tablet 4bac9516c95a460a8168a10d45116e65 (DEFAULT_TABLE table=test-table [id=2c533b96bc3948c59a87fd642df56d25]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:25.192195 22235 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bac9516c95a460a8168a10d45116e65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:25.192196 22101 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bac9516c95a460a8168a10d45116e65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:25.192292 22369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bac9516c95a460a8168a10d45116e65. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:25.195299 22452 tablet_bootstrap.cc:492] T 4bac9516c95a460a8168a10d45116e65 P d8e1d70b01aa437d9d3faf6135784a6d: Bootstrap starting.
I20260619 16:58:25.195748 22453 tablet_bootstrap.cc:492] T 4bac9516c95a460a8168a10d45116e65 P e54366b1d44e45ed907a5936a1ac6c65: Bootstrap starting.
I20260619 16:58:25.196311 22452 tablet_bootstrap.cc:654] T 4bac9516c95a460a8168a10d45116e65 P d8e1d70b01aa437d9d3faf6135784a6d: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:25.196727 22453 tablet_bootstrap.cc:654] T 4bac9516c95a460a8168a10d45116e65 P e54366b1d44e45ed907a5936a1ac6c65: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:25.197109 22452 log.cc:824] T 4bac9516c95a460a8168a10d45116e65 P d8e1d70b01aa437d9d3faf6135784a6d: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:25.197448 22453 log.cc:824] T 4bac9516c95a460a8168a10d45116e65 P e54366b1d44e45ed907a5936a1ac6c65: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:25.197876 22454 tablet_bootstrap.cc:492] T 4bac9516c95a460a8168a10d45116e65 P 21fd7d5cb70a4eff9d89bceacd0a4267: Bootstrap starting.
I20260619 16:58:25.198088 22452 tablet_bootstrap.cc:492] T 4bac9516c95a460a8168a10d45116e65 P d8e1d70b01aa437d9d3faf6135784a6d: No bootstrap required, opened a new log
I20260619 16:58:25.198179 22452 ts_tablet_manager.cc:1397] T 4bac9516c95a460a8168a10d45116e65 P d8e1d70b01aa437d9d3faf6135784a6d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 16:58:25.198328 22453 tablet_bootstrap.cc:492] T 4bac9516c95a460a8168a10d45116e65 P e54366b1d44e45ed907a5936a1ac6c65: No bootstrap required, opened a new log
I20260619 16:58:25.198407 22453 ts_tablet_manager.cc:1397] T 4bac9516c95a460a8168a10d45116e65 P e54366b1d44e45ed907a5936a1ac6c65: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 16:58:25.198637 22454 tablet_bootstrap.cc:654] T 4bac9516c95a460a8168a10d45116e65 P 21fd7d5cb70a4eff9d89bceacd0a4267: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:25.199326 22454 log.cc:824] T 4bac9516c95a460a8168a10d45116e65 P 21fd7d5cb70a4eff9d89bceacd0a4267: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:25.200220 22454 tablet_bootstrap.cc:492] T 4bac9516c95a460a8168a10d45116e65 P 21fd7d5cb70a4eff9d89bceacd0a4267: No bootstrap required, opened a new log
I20260619 16:58:25.200304 22454 ts_tablet_manager.cc:1397] T 4bac9516c95a460a8168a10d45116e65 P 21fd7d5cb70a4eff9d89bceacd0a4267: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 16:58:25.200407 22453 raft_consensus.cc:348] T 4bac9516c95a460a8168a10d45116e65 P e54366b1d44e45ed907a5936a1ac6c65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54366b1d44e45ed907a5936a1ac6c65" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 33955 } } peers { permanent_uuid: "21fd7d5cb70a4eff9d89bceacd0a4267" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42927 } } peers { permanent_uuid: "d8e1d70b01aa437d9d3faf6135784a6d" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 37653 } }
I20260619 16:58:25.200592 22453 raft_consensus.cc:374] T 4bac9516c95a460a8168a10d45116e65 P e54366b1d44e45ed907a5936a1ac6c65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:25.200635 22453 raft_consensus.cc:729] T 4bac9516c95a460a8168a10d45116e65 P e54366b1d44e45ed907a5936a1ac6c65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e54366b1d44e45ed907a5936a1ac6c65, State: Initialized, Role: FOLLOWER
I20260619 16:58:25.200768 22453 consensus_queue.cc:260] T 4bac9516c95a460a8168a10d45116e65 P e54366b1d44e45ed907a5936a1ac6c65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54366b1d44e45ed907a5936a1ac6c65" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 33955 } } peers { permanent_uuid: "21fd7d5cb70a4eff9d89bceacd0a4267" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42927 } } peers { permanent_uuid: "d8e1d70b01aa437d9d3faf6135784a6d" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 37653 } }
I20260619 16:58:25.200816 22452 raft_consensus.cc:348] T 4bac9516c95a460a8168a10d45116e65 P d8e1d70b01aa437d9d3faf6135784a6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54366b1d44e45ed907a5936a1ac6c65" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 33955 } } peers { permanent_uuid: "21fd7d5cb70a4eff9d89bceacd0a4267" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42927 } } peers { permanent_uuid: "d8e1d70b01aa437d9d3faf6135784a6d" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 37653 } }
I20260619 16:58:25.201023 22452 raft_consensus.cc:374] T 4bac9516c95a460a8168a10d45116e65 P d8e1d70b01aa437d9d3faf6135784a6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:25.201081 22453 ts_tablet_manager.cc:1428] T 4bac9516c95a460a8168a10d45116e65 P e54366b1d44e45ed907a5936a1ac6c65: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 16:58:25.201071 22452 raft_consensus.cc:729] T 4bac9516c95a460a8168a10d45116e65 P d8e1d70b01aa437d9d3faf6135784a6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8e1d70b01aa437d9d3faf6135784a6d, State: Initialized, Role: FOLLOWER
I20260619 16:58:25.201203 22167 heartbeater.cc:503] Master 127.20.231.190:34643 was elected leader, sending a full tablet report...
I20260619 16:58:25.201229 22452 consensus_queue.cc:260] T 4bac9516c95a460a8168a10d45116e65 P d8e1d70b01aa437d9d3faf6135784a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54366b1d44e45ed907a5936a1ac6c65" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 33955 } } peers { permanent_uuid: "21fd7d5cb70a4eff9d89bceacd0a4267" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42927 } } peers { permanent_uuid: "d8e1d70b01aa437d9d3faf6135784a6d" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 37653 } }
I20260619 16:58:25.201539 22452 ts_tablet_manager.cc:1428] T 4bac9516c95a460a8168a10d45116e65 P d8e1d70b01aa437d9d3faf6135784a6d: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260619 16:58:25.201545 22435 heartbeater.cc:503] Master 127.20.231.190:34643 was elected leader, sending a full tablet report...
I20260619 16:58:25.202905 22454 raft_consensus.cc:348] T 4bac9516c95a460a8168a10d45116e65 P 21fd7d5cb70a4eff9d89bceacd0a4267 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54366b1d44e45ed907a5936a1ac6c65" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 33955 } } peers { permanent_uuid: "21fd7d5cb70a4eff9d89bceacd0a4267" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42927 } } peers { permanent_uuid: "d8e1d70b01aa437d9d3faf6135784a6d" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 37653 } }
I20260619 16:58:25.203075 22454 raft_consensus.cc:374] T 4bac9516c95a460a8168a10d45116e65 P 21fd7d5cb70a4eff9d89bceacd0a4267 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:25.203107 22454 raft_consensus.cc:729] T 4bac9516c95a460a8168a10d45116e65 P 21fd7d5cb70a4eff9d89bceacd0a4267 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21fd7d5cb70a4eff9d89bceacd0a4267, State: Initialized, Role: FOLLOWER
I20260619 16:58:25.203222 22454 consensus_queue.cc:260] T 4bac9516c95a460a8168a10d45116e65 P 21fd7d5cb70a4eff9d89bceacd0a4267 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54366b1d44e45ed907a5936a1ac6c65" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 33955 } } peers { permanent_uuid: "21fd7d5cb70a4eff9d89bceacd0a4267" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42927 } } peers { permanent_uuid: "d8e1d70b01aa437d9d3faf6135784a6d" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 37653 } }
I20260619 16:58:25.203552 22454 ts_tablet_manager.cc:1428] T 4bac9516c95a460a8168a10d45116e65 P 21fd7d5cb70a4eff9d89bceacd0a4267: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260619 16:58:25.203560 22301 heartbeater.cc:503] Master 127.20.231.190:34643 was elected leader, sending a full tablet report...
W20260619 16:58:25.225768 22302 tablet.cc:2369] T 4bac9516c95a460a8168a10d45116e65 P 21fd7d5cb70a4eff9d89bceacd0a4267: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 16:58:25.291093 22168 tablet.cc:2369] T 4bac9516c95a460a8168a10d45116e65 P e54366b1d44e45ed907a5936a1ac6c65: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 16:58:25.398557 22436 tablet.cc:2369] T 4bac9516c95a460a8168a10d45116e65 P d8e1d70b01aa437d9d3faf6135784a6d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:58:25.548460 22458 raft_consensus.cc:482] T 4bac9516c95a460a8168a10d45116e65 P e54366b1d44e45ed907a5936a1ac6c65 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:58:25.548579 22458 raft_consensus.cc:504] T 4bac9516c95a460a8168a10d45116e65 P e54366b1d44e45ed907a5936a1ac6c65 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54366b1d44e45ed907a5936a1ac6c65" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 33955 } } peers { permanent_uuid: "21fd7d5cb70a4eff9d89bceacd0a4267" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42927 } } peers { permanent_uuid: "d8e1d70b01aa437d9d3faf6135784a6d" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 37653 } }
I20260619 16:58:25.549013 22458 leader_election.cc:288] T 4bac9516c95a460a8168a10d45116e65 P e54366b1d44e45ed907a5936a1ac6c65 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21fd7d5cb70a4eff9d89bceacd0a4267 (127.20.231.130:42927), d8e1d70b01aa437d9d3faf6135784a6d (127.20.231.131:37653)
I20260619 16:58:25.553133 22449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:25.549250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:33645 (local address 127.20.231.130:42927)
0619 16:58:25.549302 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:25.549304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:25.549436 (+ 132us) server_negotiation.cc:407] Connection header received
0619 16:58:25.549478 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:25.549479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:25.549489 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:25.549557 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:25.549805 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:25.550606 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:25.552112 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:25.552690 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:25.552775 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:25.552783 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:25.553009 (+ 226us) server_negotiation.cc:299] Negotiation successful
0619 16:58:25.553057 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 16:58:25.553130 22450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:25.549399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:53875 (local address 127.20.231.131:37653)
0619 16:58:25.549449 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:25.549451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:25.549462 (+ 11us) server_negotiation.cc:407] Connection header received
0619 16:58:25.549485 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:25.549486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:25.549494 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:25.549557 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:25.549834 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:25.550563 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:25.552112 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:25.552696 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:25.552775 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:25.552783 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:25.553008 (+ 225us) server_negotiation.cc:299] Negotiation successful
0619 16:58:25.553057 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 16:58:25.553124 22461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:25.549222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:42927 (local address 127.20.231.129:33645)
0619 16:58:25.549355 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:25.549363 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:25.549423 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:25.549583 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:25.549584 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:25.549587 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:25.549679 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:25.549680 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:25.550697 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:25.550699 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:25.551997 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:25.552000 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:25.552793 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:25.552794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:25.552883 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:25.552888 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 16:58:25.552953 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 16:58:25.553005 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":23,"thread_start_us":59,"threads_started":1}
I20260619 16:58:25.553133 22462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:25.549211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:37653 (local address 127.20.231.129:53875)
0619 16:58:25.549327 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:25.549353 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:25.549423 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:25.549583 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:25.549584 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:25.549586 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:25.549674 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:25.549676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:25.550752 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:25.550753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:25.551997 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:25.552000 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:25.552791 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:25.552793 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:25.552883 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:25.552889 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 16:58:25.552953 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 16:58:25.553005 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96,"mutex_wait_us":9,"thread_start_us":71,"threads_started":1}
I20260619 16:58:25.553531 22255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bac9516c95a460a8168a10d45116e65" candidate_uuid: "e54366b1d44e45ed907a5936a1ac6c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21fd7d5cb70a4eff9d89bceacd0a4267" is_pre_election: true
I20260619 16:58:25.553735 22255 raft_consensus.cc:2454] T 4bac9516c95a460a8168a10d45116e65 P 21fd7d5cb70a4eff9d89bceacd0a4267 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e54366b1d44e45ed907a5936a1ac6c65 in term 0.
I20260619 16:58:25.553896 22389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bac9516c95a460a8168a10d45116e65" candidate_uuid: "e54366b1d44e45ed907a5936a1ac6c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8e1d70b01aa437d9d3faf6135784a6d" is_pre_election: true
I20260619 16:58:25.554009 22057 leader_election.cc:302] T 4bac9516c95a460a8168a10d45116e65 P e54366b1d44e45ed907a5936a1ac6c65 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21fd7d5cb70a4eff9d89bceacd0a4267, e54366b1d44e45ed907a5936a1ac6c65; no voters:
I20260619 16:58:25.554065 22389 raft_consensus.cc:2454] T 4bac9516c95a460a8168a10d45116e65 P d8e1d70b01aa437d9d3faf6135784a6d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e54366b1d44e45ed907a5936a1ac6c65 in term 0.
I20260619 16:58:25.554184 22458 raft_consensus.cc:2781] T 4bac9516c95a460a8168a10d45116e65 P e54366b1d44e45ed907a5936a1ac6c65 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:58:25.554280 22458 raft_consensus.cc:482] T 4bac9516c95a460a8168a10d45116e65 P e54366b1d44e45ed907a5936a1ac6c65 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:58:25.554322 22458 raft_consensus.cc:3037] T 4bac9516c95a460a8168a10d45116e65 P e54366b1d44e45ed907a5936a1ac6c65 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:25.555271 22458 raft_consensus.cc:504] T 4bac9516c95a460a8168a10d45116e65 P e54366b1d44e45ed907a5936a1ac6c65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54366b1d44e45ed907a5936a1ac6c65" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 33955 } } peers { permanent_uuid: "21fd7d5cb70a4eff9d89bceacd0a4267" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42927 } } peers { permanent_uuid: "d8e1d70b01aa437d9d3faf6135784a6d" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 37653 } }
I20260619 16:58:25.555444 22458 leader_election.cc:288] T 4bac9516c95a460a8168a10d45116e65 P e54366b1d44e45ed907a5936a1ac6c65 [CANDIDATE]: Term 1 election: Requested vote from peers 21fd7d5cb70a4eff9d89bceacd0a4267 (127.20.231.130:42927), d8e1d70b01aa437d9d3faf6135784a6d (127.20.231.131:37653)
I20260619 16:58:25.555658 22255 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bac9516c95a460a8168a10d45116e65" candidate_uuid: "e54366b1d44e45ed907a5936a1ac6c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21fd7d5cb70a4eff9d89bceacd0a4267"
I20260619 16:58:25.555681 22389 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bac9516c95a460a8168a10d45116e65" candidate_uuid: "e54366b1d44e45ed907a5936a1ac6c65" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8e1d70b01aa437d9d3faf6135784a6d"
I20260619 16:58:25.555785 22255 raft_consensus.cc:3037] T 4bac9516c95a460a8168a10d45116e65 P 21fd7d5cb70a4eff9d89bceacd0a4267 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:25.555780 22389 raft_consensus.cc:3037] T 4bac9516c95a460a8168a10d45116e65 P d8e1d70b01aa437d9d3faf6135784a6d [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:25.556565 22389 raft_consensus.cc:2454] T 4bac9516c95a460a8168a10d45116e65 P d8e1d70b01aa437d9d3faf6135784a6d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e54366b1d44e45ed907a5936a1ac6c65 in term 1.
I20260619 16:58:25.556560 22255 raft_consensus.cc:2454] T 4bac9516c95a460a8168a10d45116e65 P 21fd7d5cb70a4eff9d89bceacd0a4267 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e54366b1d44e45ed907a5936a1ac6c65 in term 1.
I20260619 16:58:25.556777 22054 leader_election.cc:302] T 4bac9516c95a460a8168a10d45116e65 P e54366b1d44e45ed907a5936a1ac6c65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8e1d70b01aa437d9d3faf6135784a6d, e54366b1d44e45ed907a5936a1ac6c65; no voters:
I20260619 16:58:25.556885 22458 raft_consensus.cc:2781] T 4bac9516c95a460a8168a10d45116e65 P e54366b1d44e45ed907a5936a1ac6c65 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:25.557075 22458 raft_consensus.cc:686] T 4bac9516c95a460a8168a10d45116e65 P e54366b1d44e45ed907a5936a1ac6c65 [term 1 LEADER]: Becoming Leader. State: Replica: e54366b1d44e45ed907a5936a1ac6c65, State: Running, Role: LEADER
I20260619 16:58:25.557214 22458 consensus_queue.cc:237] T 4bac9516c95a460a8168a10d45116e65 P e54366b1d44e45ed907a5936a1ac6c65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54366b1d44e45ed907a5936a1ac6c65" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 33955 } } peers { permanent_uuid: "21fd7d5cb70a4eff9d89bceacd0a4267" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42927 } } peers { permanent_uuid: "d8e1d70b01aa437d9d3faf6135784a6d" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 37653 } }
I20260619 16:58:25.558091 21976 catalog_manager.cc:5495] T 4bac9516c95a460a8168a10d45116e65 P e54366b1d44e45ed907a5936a1ac6c65 reported cstate change: term changed from 0 to 1, leader changed from <none> to e54366b1d44e45ed907a5936a1ac6c65 (127.20.231.129). New cstate: current_term: 1 leader_uuid: "e54366b1d44e45ed907a5936a1ac6c65" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54366b1d44e45ed907a5936a1ac6c65" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 33955 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "21fd7d5cb70a4eff9d89bceacd0a4267" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42927 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d8e1d70b01aa437d9d3faf6135784a6d" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 37653 } health_report { overall_health: UNKNOWN } } }
Jun 19 16:58:25 dist-test-slave-vnbj krb5kdc[21931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.129@KRBTEST.COM
I20260619 16:58:25.639189 22451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:25.629680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41882 (local address 127.20.231.129:33955)
0619 16:58:25.629729 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:25.629732 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:25.629891 (+ 159us) server_negotiation.cc:407] Connection header received
0619 16:58:25.629970 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:25.629971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:25.629983 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:25.630042 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:25.630632 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:25.631285 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:25.632037 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:25.632232 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:25.632319 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:25.635466 (+ 3147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:25.635481 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:25.635481 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:25.635491 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:25.637762 (+ 2271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:25.638242 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:25.638246 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:25.638247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:25.638333 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:25.638541 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:25.638542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:25.638543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:25.638708 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:25.638897 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:25.639009 (+ 112us) server_negotiation.cc:299] Negotiation successful
0619 16:58:25.639105 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Jun 19 16:58:25 dist-test-slave-vnbj krb5kdc[21931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888305, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 16:58:25 dist-test-slave-vnbj krb5kdc[21931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888305, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.231.129@KRBTEST.COM
I20260619 16:58:25.665174 22451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:25.657024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41894 (local address 127.20.231.129:33955)
0619 16:58:25.657125 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:25.657127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:25.657244 (+ 117us) server_negotiation.cc:407] Connection header received
0619 16:58:25.657297 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:25.657298 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:25.657307 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:25.657358 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:25.657961 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:25.658438 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:25.659332 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:25.659539 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:25.659745 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:25.662245 (+ 2500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:25.662247 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:25.662247 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:25.662255 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:25.664063 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:25.664482 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:25.664483 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:25.664483 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:25.664536 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:25.664695 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:25.664696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:25.664696 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:25.664794 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:25.664886 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:25.665008 (+ 122us) server_negotiation.cc:299] Negotiation successful
0619 16:58:25.665069 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
Jun 19 16:58:25 dist-test-slave-vnbj krb5kdc[21931](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 16:58:25 dist-test-slave-vnbj krb5kdc[21931](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888305, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.129@KRBTEST.COM
I20260619 16:58:25.689766 22451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:25.681715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41904 (local address 127.20.231.129:33955)
0619 16:58:25.681778 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:25.681780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:25.681950 (+ 170us) server_negotiation.cc:407] Connection header received
0619 16:58:25.682043 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:25.682044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:25.682054 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:25.682115 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:25.682707 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:25.683259 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:25.684048 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:25.684246 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:25.684337 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:25.687094 (+ 2757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:25.687125 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:25.687126 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:25.687135 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:25.688453 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:25.688934 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:25.688935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:25.688936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:25.688998 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:25.689201 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:25.689202 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:25.689202 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:25.689309 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:25.689427 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:25.689571 (+ 144us) server_negotiation.cc:299] Negotiation successful
0619 16:58:25.689658 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260619 16:58:25.690332 22081 tablet_service.cc:2908] Rejecting scan request for tablet 4bac9516c95a460a8168a10d45116e65: Uninitialized: clean time has not yet been initialized
I20260619 16:58:25.691345 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 22038
I20260619 16:58:25.696727 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 22172
I20260619 16:58:25.701272 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 22306
I20260619 16:58:25.705994 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 21944
2026-06-19T16:58:25Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2415 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:58:25 dist-test-slave-vnbj krb5kdc[22477](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:58:25 dist-test-slave-vnbj krb5kdc[22477](info): set up 2 sockets
Jun 19 16:58:25 dist-test-slave-vnbj krb5kdc[22477](info): commencing operation
krb5kdc: starting...
W20260619 16:58:26.854720 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 16:58:26 dist-test-slave-vnbj krb5kdc[22477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:58:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:58:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.231.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
I20260619 16:58:26.981576 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:33825
--webserver_interface=127.20.231.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:38763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:33825
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:27.064558 22490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:27.064778 22490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:27.064801 22490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:27.066025 22490 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:27.066071 22490 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 16:58:27.066085 22490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:27.066098 22490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:27.066110 22490 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:27.066123 22490 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:58:27.067728 22490 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:38763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:33825
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:33825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22490
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:27.067993 22490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:27.068235 22490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:27.073393 22496 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:27.073413 22498 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:27.073422 22490 server_base.cc:1034] running on GCE node
W20260619 16:58:27.073400 22495 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:27.074003 22490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:27.074352 22490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:27.075533 22490 hybrid_clock.cc:648] HybridClock initialized: now 1781888307075512 us; error 45 us; skew 500 ppm
Jun 19 16:58:27 dist-test-slave-vnbj krb5kdc[22477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888307, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:27.086376 22490 init.cc:377] Logged in from keytab as kudu/127.20.231.190@KRBTEST.COM (short username kudu)
I20260619 16:58:27.087010 22490 webserver.cc:460] Webserver started at http://127.20.231.190:43621/ using document root <none> and password file <none>
I20260619 16:58:27.087282 22490 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:27.087358 22490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:27.087512 22490 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:27.088667 22490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "e9440f86afd64d71bad59055d60cb3a2"
format_stamp: "Formatted at 2026-06-19 16:58:27 on dist-test-slave-vnbj"
server_key: "859d0bdffc30d06609fe93476f58f636"
server_key_iv: "749490e75b4678baac3d79223381c828"
server_key_version: "encryptionkey@0"
I20260619 16:58:27.089066 22490 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "e9440f86afd64d71bad59055d60cb3a2"
format_stamp: "Formatted at 2026-06-19 16:58:27 on dist-test-slave-vnbj"
server_key: "859d0bdffc30d06609fe93476f58f636"
server_key_iv: "749490e75b4678baac3d79223381c828"
server_key_version: "encryptionkey@0"
I20260619 16:58:27.090586 22490 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:58:27.091485 22505 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:27.091662 22490 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 16:58:27.091818 22490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "e9440f86afd64d71bad59055d60cb3a2"
format_stamp: "Formatted at 2026-06-19 16:58:27 on dist-test-slave-vnbj"
server_key: "859d0bdffc30d06609fe93476f58f636"
server_key_iv: "749490e75b4678baac3d79223381c828"
server_key_version: "encryptionkey@0"
I20260619 16:58:27.091907 22490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:27.109138 22490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:27.111744 22490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:27.111934 22490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:27.117254 22490 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:33825
I20260619 16:58:27.117321 22567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:33825 every 8 connection(s)
I20260619 16:58:27.117661 22490 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 16:58:27.118296 22568 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:27.120761 22568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9440f86afd64d71bad59055d60cb3a2: Bootstrap starting.
I20260619 16:58:27.121359 22568 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9440f86afd64d71bad59055d60cb3a2: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:27.121659 22568 log.cc:824] T 00000000000000000000000000000000 P e9440f86afd64d71bad59055d60cb3a2: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:27.122406 22568 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9440f86afd64d71bad59055d60cb3a2: No bootstrap required, opened a new log
I20260619 16:58:27.124702 22568 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9440f86afd64d71bad59055d60cb3a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9440f86afd64d71bad59055d60cb3a2" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 33825 } }
I20260619 16:58:27.124850 22568 raft_consensus.cc:374] T 00000000000000000000000000000000 P e9440f86afd64d71bad59055d60cb3a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:27.124872 22568 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9440f86afd64d71bad59055d60cb3a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9440f86afd64d71bad59055d60cb3a2, State: Initialized, Role: FOLLOWER
I20260619 16:58:27.125028 22568 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9440f86afd64d71bad59055d60cb3a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9440f86afd64d71bad59055d60cb3a2" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 33825 } }
I20260619 16:58:27.125118 22568 raft_consensus.cc:388] T 00000000000000000000000000000000 P e9440f86afd64d71bad59055d60cb3a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:58:27.125166 22568 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9440f86afd64d71bad59055d60cb3a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:58:27.125218 22568 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9440f86afd64d71bad59055d60cb3a2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:27.125823 22568 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9440f86afd64d71bad59055d60cb3a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9440f86afd64d71bad59055d60cb3a2" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 33825 } }
I20260619 16:58:27.125913 22568 leader_election.cc:302] T 00000000000000000000000000000000 P e9440f86afd64d71bad59055d60cb3a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e9440f86afd64d71bad59055d60cb3a2; no voters:
I20260619 16:58:27.126140 22568 leader_election.cc:288] T 00000000000000000000000000000000 P e9440f86afd64d71bad59055d60cb3a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:58:27.126191 22571 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9440f86afd64d71bad59055d60cb3a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:27.126415 22568 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9440f86afd64d71bad59055d60cb3a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:27.126546 22571 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9440f86afd64d71bad59055d60cb3a2 [term 1 LEADER]: Becoming Leader. State: Replica: e9440f86afd64d71bad59055d60cb3a2, State: Running, Role: LEADER
I20260619 16:58:27.126648 22571 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9440f86afd64d71bad59055d60cb3a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9440f86afd64d71bad59055d60cb3a2" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 33825 } }
I20260619 16:58:27.126932 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 22490
I20260619 16:58:27.127054 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 16:58:27.127106 22573 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9440f86afd64d71bad59055d60cb3a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9440f86afd64d71bad59055d60cb3a2. Latest consensus state: current_term: 1 leader_uuid: "e9440f86afd64d71bad59055d60cb3a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9440f86afd64d71bad59055d60cb3a2" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 33825 } } }
I20260619 16:58:27.127202 22573 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9440f86afd64d71bad59055d60cb3a2 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:27.127252 21406 external_mini_cluster.cc:1428] Setting key afb721f5d61afa4c23d4b96d4572dc1c
I20260619 16:58:27.127156 22572 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9440f86afd64d71bad59055d60cb3a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e9440f86afd64d71bad59055d60cb3a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e9440f86afd64d71bad59055d60cb3a2" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 33825 } } }
I20260619 16:58:27.127408 22572 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9440f86afd64d71bad59055d60cb3a2 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:27.127871 22575 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:58:27.128453 22575 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:58:27.130231 22575 catalog_manager.cc:1347] Generated new cluster ID: bbc433be8a074e38ab008d9e7a9c159a
I20260619 16:58:27.130285 22575 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 16:58:27 dist-test-slave-vnbj krb5kdc[22477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888306, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:27.138659 22583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:27.127946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51984 (local address 127.20.231.190:33825)
0619 16:58:27.128110 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:27.128121 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:27.128158 (+ 37us) server_negotiation.cc:407] Connection header received
0619 16:58:27.128297 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:27.128305 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:27.128366 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:27.129076 (+ 710us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:27.129285 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.130672 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.131601 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.132079 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.132226 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:27.134934 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:27.134939 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:27.134940 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:27.134953 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:27.136908 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:27.137348 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:27.137351 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:27.137352 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:27.137470 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:27.137705 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:27.137706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:27.137706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:27.137985 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:27.138172 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:27.138273 (+ 101us) server_negotiation.cc:299] Negotiation successful
0619 16:58:27.138443 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":92,"threads_started":1}
I20260619 16:58:27.139583 22575 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:58:27.140272 22575 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:58:27.145805 22575 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e9440f86afd64d71bad59055d60cb3a2: Generated new TSK 0
I20260619 16:58:27.145992 22575 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.231.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab.
Entry for principal kudu/127.20.231.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab.
I20260619 16:58:27.178045 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.129:0
--local_ip_for_outbound_sockets=127.20.231.129
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:33825
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:38763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:27.266183 22594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:27.266409 22594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:27.266434 22594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:27.267663 22594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:27.267711 22594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:27.267732 22594 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 16:58:27.267750 22594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.129
I20260619 16:58:27.269227 22594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:38763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.231.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.231.190:33825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.129
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:27.269537 22594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:27.269809 22594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:27.275144 22599 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:27.275177 22602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:27.275281 22594 server_base.cc:1034] running on GCE node
W20260619 16:58:27.275139 22600 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:27.275628 22594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:27.275928 22594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:27.277180 22594 hybrid_clock.cc:648] HybridClock initialized: now 1781888307277160 us; error 36 us; skew 500 ppm
Jun 19 16:58:27 dist-test-slave-vnbj krb5kdc[22477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888307, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:27.280114 22594 init.cc:377] Logged in from keytab as kudu/127.20.231.129@KRBTEST.COM (short username kudu)
I20260619 16:58:27.280591 22594 webserver.cc:460] Webserver started at http://127.20.231.129:37867/ using document root <none> and password file <none>
I20260619 16:58:27.280850 22594 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:27.280923 22594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:27.281080 22594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:27.282241 22594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-0/data/instance:
uuid: "b07ee522f96d4f4db5788b2bce6d8afc"
format_stamp: "Formatted at 2026-06-19 16:58:27 on dist-test-slave-vnbj"
server_key: "39e5d08411dda599c025fb88e8948189"
server_key_iv: "f978608c7e97398ffef4b6ba5ed79be9"
server_key_version: "encryptionkey@0"
I20260619 16:58:27.282617 22594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance:
uuid: "b07ee522f96d4f4db5788b2bce6d8afc"
format_stamp: "Formatted at 2026-06-19 16:58:27 on dist-test-slave-vnbj"
server_key: "39e5d08411dda599c025fb88e8948189"
server_key_iv: "f978608c7e97398ffef4b6ba5ed79be9"
server_key_version: "encryptionkey@0"
I20260619 16:58:27.284173 22594 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:58:27.285117 22609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:27.285333 22594 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:27.285454 22594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-0/wal
uuid: "b07ee522f96d4f4db5788b2bce6d8afc"
format_stamp: "Formatted at 2026-06-19 16:58:27 on dist-test-slave-vnbj"
server_key: "39e5d08411dda599c025fb88e8948189"
server_key_iv: "f978608c7e97398ffef4b6ba5ed79be9"
server_key_version: "encryptionkey@0"
I20260619 16:58:27.285534 22594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:27.297082 22594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:27.299638 22594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:27.299858 22594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:27.300462 22594 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:27.300535 22594 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:27.300588 22594 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:27.300635 22594 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:27.308169 22594 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.129:46431
I20260619 16:58:27.308408 22724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.129:46431 every 8 connection(s)
Jun 19 16:58:27 dist-test-slave-vnbj krb5kdc[22477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888307, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:27.308647 22594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
I20260619 16:58:27.312364 22583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:27.301441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:57711 (local address 127.20.231.190:33825)
0619 16:58:27.301516 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:27.301519 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:27.301529 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:58:27.301987 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:27.301988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:27.302005 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:27.302080 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:27.303062 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.304042 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.305633 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.305988 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.306086 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:27.309116 (+ 3030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:27.309117 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:27.309118 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:27.309126 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:27.310720 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:27.311244 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:27.311245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:27.311245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:27.311298 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:27.311571 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:27.311572 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:27.311573 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:27.311713 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:27.311771 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:27.312224 (+ 453us) server_negotiation.cc:299] Negotiation successful
0619 16:58:27.312298 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 16:58:27.312557 22618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:27.301119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33825 (local address 127.20.231.129:57711)
0619 16:58:27.301310 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:27.301348 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:27.301881 (+ 533us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:27.302203 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:27.302208 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:27.302596 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:27.302944 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:27.302951 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.304155 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.304157 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:27.305511 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:27.305515 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.306116 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.306117 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:27.306275 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:27.306949 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:27.306961 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:27.308999 (+ 2038us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:27.310809 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:27.310816 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:27.310818 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:27.311170 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:27.311363 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:27.311363 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:27.311364 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:27.311473 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:27.311927 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:27.311933 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:27.312075 (+ 142us) client_negotiation.cc:769] Sending connection context
0619 16:58:27.312174 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 16:58:27.312309 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":116,"threads_started":1}
I20260619 16:58:27.313189 22725 heartbeater.cc:347] Connected to a master server at 127.20.231.190:33825
I20260619 16:58:27.313328 22725 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:27.313525 22725 heartbeater.cc:511] Master 127.20.231.190:33825 requested a full tablet report, sending...
I20260619 16:58:27.313947 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 22594
I20260619 16:58:27.314044 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance
I20260619 16:58:27.314102 22521 ts_manager.cc:194] Registered new tserver with Master: b07ee522f96d4f4db5788b2bce6d8afc (127.20.231.129:46431)
I20260619 16:58:27.314210 21406 external_mini_cluster.cc:1428] Setting key 13cffaae3bf78fb3ea0fd1a2c2beaba3
I20260619 16:58:27.315055 22521 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.129@KRBTEST.COM'} at 127.20.231.129:57711
I20260619 16:58:27.321296 22583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:27.314587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:43583 (local address 127.20.231.190:33825)
0619 16:58:27.314632 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:27.314634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:27.314811 (+ 177us) server_negotiation.cc:407] Connection header received
0619 16:58:27.314904 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:27.314904 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:27.314915 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:27.314969 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:27.315361 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.316117 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.316887 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.317095 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.317180 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:27.318236 (+ 1056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:27.318237 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:27.318238 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:27.318244 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:27.319823 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:27.320279 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:27.320280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:27.320281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:27.320337 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:27.320608 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:27.320609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:27.320609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:27.320825 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:27.320925 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:27.321148 (+ 223us) server_negotiation.cc:299] Negotiation successful
0619 16:58:27.321224 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260619 16:58:27.321331 22731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:27.314526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33825 (local address 127.20.231.129:43583)
0619 16:58:27.314731 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:27.314742 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:27.314825 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:27.314980 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:27.314982 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:27.315209 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:27.315278 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:27.315281 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.316248 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.316249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:27.316772 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:27.316774 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.317180 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.317181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:27.317266 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:27.317699 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:27.317705 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:27.318127 (+ 422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:27.319911 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:27.319912 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:27.319913 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:27.320169 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:27.320425 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:27.320427 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:27.320427 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:27.320505 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:27.320937 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:27.320938 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:27.320998 (+ 60us) client_negotiation.cc:769] Sending connection context
0619 16:58:27.321082 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 16:58:27.321205 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":150,"threads_started":1}
WARNING: no policy specified for kudu/127.20.231.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab.
Entry for principal kudu/127.20.231.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab.
I20260619 16:58:27.341372 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.130:0
--local_ip_for_outbound_sockets=127.20.231.130
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:33825
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:38763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:27.428380 22734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:27.428653 22734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:27.428683 22734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:27.429898 22734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:27.429941 22734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:27.429965 22734 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 16:58:27.429981 22734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.130
I20260619 16:58:27.431461 22734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:38763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.231.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.231.190:33825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.130
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:27.431663 22734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:27.431855 22734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:27.437019 22739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:27.437031 22740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:27.437091 22742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:27.437350 22734 server_base.cc:1034] running on GCE node
I20260619 16:58:27.437579 22734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:27.437914 22734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:27.439088 22734 hybrid_clock.cc:648] HybridClock initialized: now 1781888307439077 us; error 32 us; skew 500 ppm
Jun 19 16:58:27 dist-test-slave-vnbj krb5kdc[22477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888307, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:27.441625 22734 init.cc:377] Logged in from keytab as kudu/127.20.231.130@KRBTEST.COM (short username kudu)
I20260619 16:58:27.442314 22734 webserver.cc:460] Webserver started at http://127.20.231.130:35011/ using document root <none> and password file <none>
I20260619 16:58:27.442565 22734 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:27.442638 22734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:27.442809 22734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:27.443945 22734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-1/data/instance:
uuid: "35b7b94f6a984ce88fdf4c540fec5817"
format_stamp: "Formatted at 2026-06-19 16:58:27 on dist-test-slave-vnbj"
server_key: "01897805428063ca1f73c3fa126f2397"
server_key_iv: "55949bf7c621d75dae6387e42a7b5e05"
server_key_version: "encryptionkey@0"
I20260619 16:58:27.444315 22734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance:
uuid: "35b7b94f6a984ce88fdf4c540fec5817"
format_stamp: "Formatted at 2026-06-19 16:58:27 on dist-test-slave-vnbj"
server_key: "01897805428063ca1f73c3fa126f2397"
server_key_iv: "55949bf7c621d75dae6387e42a7b5e05"
server_key_version: "encryptionkey@0"
I20260619 16:58:27.445669 22734 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:58:27.446472 22749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:27.446660 22734 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:27.446758 22734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-1/wal
uuid: "35b7b94f6a984ce88fdf4c540fec5817"
format_stamp: "Formatted at 2026-06-19 16:58:27 on dist-test-slave-vnbj"
server_key: "01897805428063ca1f73c3fa126f2397"
server_key_iv: "55949bf7c621d75dae6387e42a7b5e05"
server_key_version: "encryptionkey@0"
I20260619 16:58:27.446841 22734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:27.475351 22734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:27.477898 22734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:27.478082 22734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:27.478720 22734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:27.478787 22734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:27.478830 22734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:27.478888 22734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 16:58:27 dist-test-slave-vnbj krb5kdc[22477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888307, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:27.486279 22734 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.130:33811
I20260619 16:58:27.486649 22734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
I20260619 16:58:27.486697 22864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.130:33811 every 8 connection(s)
I20260619 16:58:27.487277 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 22734
I20260619 16:58:27.487373 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance
I20260619 16:58:27.487555 21406 external_mini_cluster.cc:1428] Setting key 2ba3522f68aa49e03559e9d0384509bd
I20260619 16:58:27.489508 22758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:27.479170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33825 (local address 127.20.231.130:40039)
0619 16:58:27.479329 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:27.479366 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:27.479888 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:27.480163 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:27.480168 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:27.480491 (+ 323us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:27.480695 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:27.480700 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.481435 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.481437 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:27.482292 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:27.482295 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.482714 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.482715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:27.482903 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:27.483593 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:27.483607 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:27.485941 (+ 2334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:27.487926 (+ 1985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:27.487933 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:27.487935 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:27.488242 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:27.488479 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:27.488479 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:27.488480 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:27.488655 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:27.488967 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:27.488971 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:27.489093 (+ 122us) client_negotiation.cc:769] Sending connection context
0619 16:58:27.489214 (+ 121us) client_negotiation.cc:240] Negotiation successful
0619 16:58:27.489351 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":80,"threads_started":1}
I20260619 16:58:27.489501 22583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:27.479197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:40039 (local address 127.20.231.190:33825)
0619 16:58:27.479256 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:27.479258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:27.479450 (+ 192us) server_negotiation.cc:407] Connection header received
0619 16:58:27.480005 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:27.480006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:27.480028 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:27.480080 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:27.480792 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.481335 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.482395 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.482603 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.482701 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:27.486050 (+ 3349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:27.486052 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:27.486053 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:27.486062 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:27.487824 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:27.488342 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:27.488344 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:27.488345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:27.488409 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:27.488726 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:27.488727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:27.488728 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:27.488899 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:27.488972 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:27.489262 (+ 290us) server_negotiation.cc:299] Negotiation successful
0619 16:58:27.489362 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 16:58:27.490149 22865 heartbeater.cc:347] Connected to a master server at 127.20.231.190:33825
I20260619 16:58:27.490243 22865 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:27.490446 22865 heartbeater.cc:511] Master 127.20.231.190:33825 requested a full tablet report, sending...
I20260619 16:58:27.490906 22521 ts_manager.cc:194] Registered new tserver with Master: 35b7b94f6a984ce88fdf4c540fec5817 (127.20.231.130:33811)
I20260619 16:58:27.491346 22521 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.130@KRBTEST.COM'} at 127.20.231.130:40039
WARNING: no policy specified for kudu/127.20.231.131@KRBTEST.COM; defaulting to no policy
I20260619 16:58:27.498027 22872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:27.491554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33825 (local address 127.20.231.130:50333)
0619 16:58:27.491703 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:27.491714 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:27.491778 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:27.491895 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:27.491895 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:27.492044 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:27.492093 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:27.492094 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.493032 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.493034 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:27.493570 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:27.493572 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.494115 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.494116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:27.494173 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:27.494676 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:27.494680 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:27.495032 (+ 352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:27.496824 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:27.496825 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:27.496825 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:27.496987 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:27.497222 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:27.497222 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:27.497223 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:27.497285 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:27.497668 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:27.497669 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:27.497718 (+ 49us) client_negotiation.cc:769] Sending connection context
0619 16:58:27.497783 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 16:58:27.497898 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":81,"threads_started":1}
I20260619 16:58:27.498020 22583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:27.491460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:50333 (local address 127.20.231.190:33825)
0619 16:58:27.491503 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:27.491506 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:27.491791 (+ 285us) server_negotiation.cc:407] Connection header received
0619 16:58:27.491816 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:27.491817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:27.491827 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:27.491895 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:27.492166 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.492893 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.493666 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.494028 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.494111 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:27.495121 (+ 1010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:27.495122 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:27.495123 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:27.495130 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:27.496739 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:27.497063 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:27.497064 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:27.497064 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:27.497123 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:27.497368 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:27.497369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:27.497369 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:27.497579 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:27.497663 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:27.497834 (+ 171us) server_negotiation.cc:299] Negotiation successful
0619 16:58:27.497918 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab.
Entry for principal kudu/127.20.231.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab.
I20260619 16:58:27.518003 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.131:0
--local_ip_for_outbound_sockets=127.20.231.131
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:33825
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:38763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:27.601534 22874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:27.601758 22874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:27.601785 22874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:27.603029 22874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:27.603073 22874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:27.603096 22874 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 16:58:27.603113 22874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.131
I20260619 16:58:27.604562 22874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:38763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.231.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.231.190:33825
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.131
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:27.604813 22874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:27.605064 22874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:27.610256 22879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:27.610229 22882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:27.610229 22880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:27.610736 22874 server_base.cc:1034] running on GCE node
I20260619 16:58:27.610996 22874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:27.611233 22874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:27.612389 22874 hybrid_clock.cc:648] HybridClock initialized: now 1781888307612378 us; error 27 us; skew 500 ppm
Jun 19 16:58:27 dist-test-slave-vnbj krb5kdc[22477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888307, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:27.615463 22874 init.cc:377] Logged in from keytab as kudu/127.20.231.131@KRBTEST.COM (short username kudu)
I20260619 16:58:27.616019 22874 webserver.cc:460] Webserver started at http://127.20.231.131:40677/ using document root <none> and password file <none>
I20260619 16:58:27.616263 22874 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:27.616333 22874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:27.616487 22874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:27.617575 22874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-2/data/instance:
uuid: "89b668bda708492c8caf7dcbaad7f693"
format_stamp: "Formatted at 2026-06-19 16:58:27 on dist-test-slave-vnbj"
server_key: "cc541b7b9277c625a972e4a8740ed96d"
server_key_iv: "9df85b9fb6fdec26a8712fca8fde0488"
server_key_version: "encryptionkey@0"
I20260619 16:58:27.617954 22874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance:
uuid: "89b668bda708492c8caf7dcbaad7f693"
format_stamp: "Formatted at 2026-06-19 16:58:27 on dist-test-slave-vnbj"
server_key: "cc541b7b9277c625a972e4a8740ed96d"
server_key_iv: "9df85b9fb6fdec26a8712fca8fde0488"
server_key_version: "encryptionkey@0"
I20260619 16:58:27.619510 22874 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:58:27.620440 22889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:27.620766 22874 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:58:27.620916 22874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-2/wal
uuid: "89b668bda708492c8caf7dcbaad7f693"
format_stamp: "Formatted at 2026-06-19 16:58:27 on dist-test-slave-vnbj"
server_key: "cc541b7b9277c625a972e4a8740ed96d"
server_key_iv: "9df85b9fb6fdec26a8712fca8fde0488"
server_key_version: "encryptionkey@0"
I20260619 16:58:27.621001 22874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:27.640854 22874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:27.644095 22874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:27.644272 22874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:27.644832 22874 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:27.644901 22874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:27.644946 22874 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:27.644973 22874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:27.652567 22874 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.131:40031
I20260619 16:58:27.652767 23004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.131:40031 every 8 connection(s)
Jun 19 16:58:27 dist-test-slave-vnbj krb5kdc[22477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888307, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:27.653004 22874 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
I20260619 16:58:27.653295 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 22874
I20260619 16:58:27.653379 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance
I20260619 16:58:27.653550 21406 external_mini_cluster.cc:1428] Setting key e67e3151b85dec0f8358ce825e24f347
I20260619 16:58:27.656482 22583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:27.645400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:60391 (local address 127.20.231.190:33825)
0619 16:58:27.645448 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:27.645450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:27.645750 (+ 300us) server_negotiation.cc:407] Connection header received
0619 16:58:27.646403 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:27.646404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:27.646419 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:27.646505 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:27.647550 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.648401 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.650037 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.650417 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.650496 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:27.653506 (+ 3010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:27.653508 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:27.653508 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:27.653515 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:27.655102 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:27.655502 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:27.655502 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:27.655503 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:27.655556 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:27.655901 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:27.655902 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:27.655902 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:27.656011 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:27.656067 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:27.656357 (+ 290us) server_negotiation.cc:299] Negotiation successful
0619 16:58:27.656420 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 16:58:27.656683 22898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:27.645371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33825 (local address 127.20.231.131:60391)
0619 16:58:27.645554 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:27.645640 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:27.646294 (+ 654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:27.646604 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:27.646629 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:27.647127 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:27.647439 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:27.647446 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.648510 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.648511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:27.649926 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:27.649929 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.650563 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.650565 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:27.650721 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:27.651398 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:27.651411 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:27.653411 (+ 2000us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:27.655197 (+ 1786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:27.655203 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:27.655205 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:27.655430 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:27.655706 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:27.655708 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:27.655708 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:27.655834 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:27.656099 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:27.656105 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:27.656225 (+ 120us) client_negotiation.cc:769] Sending connection context
0619 16:58:27.656333 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 16:58:27.656461 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":84,"threads_started":1}
I20260619 16:58:27.657272 23005 heartbeater.cc:347] Connected to a master server at 127.20.231.190:33825
I20260619 16:58:27.657356 23005 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:27.657538 23005 heartbeater.cc:511] Master 127.20.231.190:33825 requested a full tablet report, sending...
I20260619 16:58:27.657900 22521 ts_manager.cc:194] Registered new tserver with Master: 89b668bda708492c8caf7dcbaad7f693 (127.20.231.131:40031)
I20260619 16:58:27.658383 22521 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.131@KRBTEST.COM'} at 127.20.231.131:60391
I20260619 16:58:27.663928 22583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:27.658397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:32895 (local address 127.20.231.190:33825)
0619 16:58:27.658442 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:27.658444 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:27.658512 (+ 68us) server_negotiation.cc:407] Connection header received
0619 16:58:27.658572 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:27.658573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:27.658583 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:27.658631 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:27.659019 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.659510 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.660214 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.660401 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.660472 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:27.661386 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:27.661387 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:27.661387 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:27.661394 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:27.662786 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:27.663155 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:27.663156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:27.663156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:27.663221 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:27.663419 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:27.663420 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:27.663420 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:27.663533 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:27.663602 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:27.663765 (+ 163us) server_negotiation.cc:299] Negotiation successful
0619 16:58:27.663838 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 16:58:27.663942 23011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:27.658334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33825 (local address 127.20.231.131:32895)
0619 16:58:27.658454 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:27.658465 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:27.658524 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:27.658633 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:27.658634 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:27.658819 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:27.658928 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:27.658930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.659601 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.659602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:27.660118 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:27.660120 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.660469 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.660470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:27.660542 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:27.660994 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:27.660999 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:27.661281 (+ 282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:27.662888 (+ 1607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:27.662890 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:27.662890 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:27.663068 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:27.663287 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:27.663288 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:27.663288 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:27.663347 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:27.663604 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:27.663605 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:27.663650 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 16:58:27.663740 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 16:58:27.663839 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":65,"threads_started":1}
I20260619 16:58:27.665362 21406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 16:58:27 dist-test-slave-vnbj krb5kdc[22477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888307, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 16:58:27 dist-test-slave-vnbj krb5kdc[22477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888307, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:27.689268 22583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:27.681361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54746 (local address 127.20.231.190:33825)
0619 16:58:27.681427 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:27.681429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:27.681438 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:58:27.681506 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:27.681507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:27.681516 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:27.681566 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:27.682221 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.682697 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.683639 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.683908 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.683983 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:27.686527 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:27.686528 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:27.686529 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:27.686536 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:27.688119 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:27.688485 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:27.688486 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:27.688486 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:27.688537 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:27.688778 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:27.688779 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:27.688779 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:27.688879 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:27.688952 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:27.689116 (+ 164us) server_negotiation.cc:299] Negotiation successful
0619 16:58:27.689178 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 16:58:27.691191 22521 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54746:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:58:27.692852 22521 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:58:27.699713 23022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:27.695169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:33811 (local address 127.0.0.1:40194)
0619 16:58:27.695328 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:27.695368 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:27.695434 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:27.696000 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:27.696001 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:27.696003 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:27.696069 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:27.696070 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.697028 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.697029 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:27.698340 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:27.698343 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.699383 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.699384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:27.699467 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:27.699482 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 16:58:27.699556 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 16:58:27.699613 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":33,"thread_start_us":76,"threads_started":1}
I20260619 16:58:27.699728 23024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:27.695273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40194 (local address 127.20.231.130:33811)
0619 16:58:27.695766 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:27.695770 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:27.695781 (+ 11us) server_negotiation.cc:407] Connection header received
0619 16:58:27.695822 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:27.695824 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:27.695844 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:27.696200 (+ 356us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:27.696259 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.696941 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.698431 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.699278 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.699376 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:27.699423 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:27.699583 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 16:58:27.699631 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":71,"threads_started":1}
I20260619 16:58:27.700121 23023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:27.695209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:46431 (local address 127.0.0.1:36204)
0619 16:58:27.695559 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:27.695571 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:27.695641 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:27.695929 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:27.695932 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:27.695934 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:27.696005 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:27.696007 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.697341 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.697342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:27.698407 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:27.698409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.699430 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.699431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:27.699554 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:27.699559 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 16:58:27.699616 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 16:58:27.699658 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":83,"threads_started":1}
I20260619 16:58:27.700464 23027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:27.695590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44620 (local address 127.20.231.131:40031)
0619 16:58:27.695855 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:27.695860 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:27.695869 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:58:27.695905 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:27.695907 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:27.695922 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:27.696009 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:27.696138 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.697247 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.699318 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.700106 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.700180 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:27.700231 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:27.700322 (+ 91us) server_negotiation.cc:299] Negotiation successful
0619 16:58:27.700370 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":81,"threads_started":1}
I20260619 16:58:27.700480 23026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:27.695386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:40031 (local address 127.0.0.1:44620)
0619 16:58:27.695624 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:27.695634 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:27.695693 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:27.696005 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:27.696006 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:27.696008 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:27.696070 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:27.696071 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.698494 (+ 2423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.698495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:27.699216 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:27.699217 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.700180 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.700181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:27.700243 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:27.700247 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:58:27.700376 (+ 129us) client_negotiation.cc:240] Negotiation successful
0619 16:58:27.700415 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":5,"thread_start_us":90,"threads_started":1}
I20260619 16:58:27.700603 22799 tablet_service.cc:1467] Processing CreateTablet for tablet e2e272e647cc4a6e8a69378a770f6821 (DEFAULT_TABLE table=test-table [id=65b9471487a84df5a5d984acff72e5f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:27.700501 23025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:27.695379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36204 (local address 127.20.231.129:46431)
0619 16:58:27.695577 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:27.695582 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:27.695635 (+ 53us) server_negotiation.cc:407] Connection header received
0619 16:58:27.695733 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:27.695735 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:27.695752 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:27.695839 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:27.696083 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.697232 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.698483 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.699248 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.699333 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:27.699395 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:27.700356 (+ 961us) server_negotiation.cc:299] Negotiation successful
0619 16:58:27.700401 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":92,"threads_started":1}
I20260619 16:58:27.700991 22799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2e272e647cc4a6e8a69378a770f6821. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:27.701112 22939 tablet_service.cc:1467] Processing CreateTablet for tablet e2e272e647cc4a6e8a69378a770f6821 (DEFAULT_TABLE table=test-table [id=65b9471487a84df5a5d984acff72e5f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:27.701406 22939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2e272e647cc4a6e8a69378a770f6821. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:27.701512 22659 tablet_service.cc:1467] Processing CreateTablet for tablet e2e272e647cc4a6e8a69378a770f6821 (DEFAULT_TABLE table=test-table [id=65b9471487a84df5a5d984acff72e5f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:27.701814 22659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2e272e647cc4a6e8a69378a770f6821. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:27.703781 23028 tablet_bootstrap.cc:492] T e2e272e647cc4a6e8a69378a770f6821 P 89b668bda708492c8caf7dcbaad7f693: Bootstrap starting.
I20260619 16:58:27.704612 23028 tablet_bootstrap.cc:654] T e2e272e647cc4a6e8a69378a770f6821 P 89b668bda708492c8caf7dcbaad7f693: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:27.704813 23029 tablet_bootstrap.cc:492] T e2e272e647cc4a6e8a69378a770f6821 P 35b7b94f6a984ce88fdf4c540fec5817: Bootstrap starting.
I20260619 16:58:27.704955 23028 log.cc:824] T e2e272e647cc4a6e8a69378a770f6821 P 89b668bda708492c8caf7dcbaad7f693: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:27.705473 23029 tablet_bootstrap.cc:654] T e2e272e647cc4a6e8a69378a770f6821 P 35b7b94f6a984ce88fdf4c540fec5817: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:27.705709 23031 tablet_bootstrap.cc:492] T e2e272e647cc4a6e8a69378a770f6821 P b07ee522f96d4f4db5788b2bce6d8afc: Bootstrap starting.
I20260619 16:58:27.705821 23029 log.cc:824] T e2e272e647cc4a6e8a69378a770f6821 P 35b7b94f6a984ce88fdf4c540fec5817: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:27.706369 23031 tablet_bootstrap.cc:654] T e2e272e647cc4a6e8a69378a770f6821 P b07ee522f96d4f4db5788b2bce6d8afc: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:27.706588 23028 tablet_bootstrap.cc:492] T e2e272e647cc4a6e8a69378a770f6821 P 89b668bda708492c8caf7dcbaad7f693: No bootstrap required, opened a new log
I20260619 16:58:27.706588 23029 tablet_bootstrap.cc:492] T e2e272e647cc4a6e8a69378a770f6821 P 35b7b94f6a984ce88fdf4c540fec5817: No bootstrap required, opened a new log
I20260619 16:58:27.706664 23029 ts_tablet_manager.cc:1397] T e2e272e647cc4a6e8a69378a770f6821 P 35b7b94f6a984ce88fdf4c540fec5817: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:58:27.706664 23028 ts_tablet_manager.cc:1397] T e2e272e647cc4a6e8a69378a770f6821 P 89b668bda708492c8caf7dcbaad7f693: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 16:58:27.706704 23031 log.cc:824] T e2e272e647cc4a6e8a69378a770f6821 P b07ee522f96d4f4db5788b2bce6d8afc: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:27.707376 23031 tablet_bootstrap.cc:492] T e2e272e647cc4a6e8a69378a770f6821 P b07ee522f96d4f4db5788b2bce6d8afc: No bootstrap required, opened a new log
I20260619 16:58:27.707458 23031 ts_tablet_manager.cc:1397] T e2e272e647cc4a6e8a69378a770f6821 P b07ee522f96d4f4db5788b2bce6d8afc: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 16:58:27.708731 23029 raft_consensus.cc:348] T e2e272e647cc4a6e8a69378a770f6821 P 35b7b94f6a984ce88fdf4c540fec5817 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } } peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } }
I20260619 16:58:27.708731 23028 raft_consensus.cc:348] T e2e272e647cc4a6e8a69378a770f6821 P 89b668bda708492c8caf7dcbaad7f693 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } } peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } }
I20260619 16:58:27.708912 23029 raft_consensus.cc:374] T e2e272e647cc4a6e8a69378a770f6821 P 35b7b94f6a984ce88fdf4c540fec5817 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:27.708912 23028 raft_consensus.cc:374] T e2e272e647cc4a6e8a69378a770f6821 P 89b668bda708492c8caf7dcbaad7f693 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:27.708813 23031 raft_consensus.cc:348] T e2e272e647cc4a6e8a69378a770f6821 P b07ee522f96d4f4db5788b2bce6d8afc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } } peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } }
I20260619 16:58:27.708948 23028 raft_consensus.cc:729] T e2e272e647cc4a6e8a69378a770f6821 P 89b668bda708492c8caf7dcbaad7f693 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89b668bda708492c8caf7dcbaad7f693, State: Initialized, Role: FOLLOWER
I20260619 16:58:27.708948 23029 raft_consensus.cc:729] T e2e272e647cc4a6e8a69378a770f6821 P 35b7b94f6a984ce88fdf4c540fec5817 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35b7b94f6a984ce88fdf4c540fec5817, State: Initialized, Role: FOLLOWER
I20260619 16:58:27.708963 23031 raft_consensus.cc:374] T e2e272e647cc4a6e8a69378a770f6821 P b07ee522f96d4f4db5788b2bce6d8afc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:27.709000 23031 raft_consensus.cc:729] T e2e272e647cc4a6e8a69378a770f6821 P b07ee522f96d4f4db5788b2bce6d8afc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b07ee522f96d4f4db5788b2bce6d8afc, State: Initialized, Role: FOLLOWER
I20260619 16:58:27.709067 23029 consensus_queue.cc:260] T e2e272e647cc4a6e8a69378a770f6821 P 35b7b94f6a984ce88fdf4c540fec5817 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } } peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } }
I20260619 16:58:27.709087 23028 consensus_queue.cc:260] T e2e272e647cc4a6e8a69378a770f6821 P 89b668bda708492c8caf7dcbaad7f693 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } } peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } }
I20260619 16:58:27.709132 23031 consensus_queue.cc:260] T e2e272e647cc4a6e8a69378a770f6821 P b07ee522f96d4f4db5788b2bce6d8afc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } } peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } }
I20260619 16:58:27.709355 23028 ts_tablet_manager.cc:1428] T e2e272e647cc4a6e8a69378a770f6821 P 89b668bda708492c8caf7dcbaad7f693: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 16:58:27.709355 23029 ts_tablet_manager.cc:1428] T e2e272e647cc4a6e8a69378a770f6821 P 35b7b94f6a984ce88fdf4c540fec5817: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 16:58:27.709384 23005 heartbeater.cc:503] Master 127.20.231.190:33825 was elected leader, sending a full tablet report...
I20260619 16:58:27.709430 23031 ts_tablet_manager.cc:1428] T e2e272e647cc4a6e8a69378a770f6821 P b07ee522f96d4f4db5788b2bce6d8afc: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 16:58:27.709672 22725 heartbeater.cc:503] Master 127.20.231.190:33825 was elected leader, sending a full tablet report...
I20260619 16:58:27.709692 22865 heartbeater.cc:503] Master 127.20.231.190:33825 was elected leader, sending a full tablet report...
W20260619 16:58:27.737360 22866 tablet.cc:2369] T e2e272e647cc4a6e8a69378a770f6821 P 35b7b94f6a984ce88fdf4c540fec5817: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 16:58:27.809538 22726 tablet.cc:2369] T e2e272e647cc4a6e8a69378a770f6821 P b07ee522f96d4f4db5788b2bce6d8afc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 16:58:27.903851 23006 tablet.cc:2369] T e2e272e647cc4a6e8a69378a770f6821 P 89b668bda708492c8caf7dcbaad7f693: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:58:27.917936 23036 raft_consensus.cc:482] T e2e272e647cc4a6e8a69378a770f6821 P b07ee522f96d4f4db5788b2bce6d8afc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:58:27.918066 23036 raft_consensus.cc:504] T e2e272e647cc4a6e8a69378a770f6821 P b07ee522f96d4f4db5788b2bce6d8afc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } } peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } }
I20260619 16:58:27.918419 23036 leader_election.cc:288] T e2e272e647cc4a6e8a69378a770f6821 P b07ee522f96d4f4db5788b2bce6d8afc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35b7b94f6a984ce88fdf4c540fec5817 (127.20.231.130:33811), 89b668bda708492c8caf7dcbaad7f693 (127.20.231.131:40031)
I20260619 16:58:27.921813 23024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:27.918781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:53865 (local address 127.20.231.130:33811)
0619 16:58:27.918894 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:27.918897 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:27.918909 (+ 12us) server_negotiation.cc:407] Connection header received
0619 16:58:27.918945 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:27.918947 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:27.918958 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:27.919043 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:27.919258 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.919813 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.921018 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.921434 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.921491 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:27.921498 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:27.921692 (+ 194us) server_negotiation.cc:299] Negotiation successful
0619 16:58:27.921742 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 16:58:27.921886 23027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:27.919040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:50695 (local address 127.20.231.131:40031)
0619 16:58:27.919095 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:27.919097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:27.919110 (+ 13us) server_negotiation.cc:407] Connection header received
0619 16:58:27.919142 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:27.919143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:27.919153 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:27.919219 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:27.919437 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.920033 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.921093 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.921510 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.921588 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:27.921597 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:27.921773 (+ 176us) server_negotiation.cc:299] Negotiation successful
0619 16:58:27.921817 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 16:58:27.921892 23037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:27.918621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:33811 (local address 127.20.231.129:53865)
0619 16:58:27.918777 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:27.918790 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:27.918871 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:27.919066 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:27.919067 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:27.919069 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:27.919156 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:27.919159 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.919926 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.919928 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:27.920922 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:27.920925 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.921526 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.921527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:27.921615 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:27.921620 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 16:58:27.921683 (+ 63us) client_negotiation.cc:240] Negotiation successful
0619 16:58:27.921720 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":4,"thread_start_us":110,"threads_started":1}
I20260619 16:58:27.922061 23038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:27.918852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:40031 (local address 127.20.231.129:50695)
0619 16:58:27.919019 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:27.919030 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:27.919083 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:27.919251 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:27.919252 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:27.919255 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:27.919319 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:27.919321 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.920145 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.920145 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:27.920957 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:27.920959 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.921595 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.921597 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:27.921684 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:27.921688 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:58:27.921746 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 16:58:27.921783 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":31,"thread_start_us":69,"threads_started":1}
I20260619 16:58:27.922279 22819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2e272e647cc4a6e8a69378a770f6821" candidate_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35b7b94f6a984ce88fdf4c540fec5817" is_pre_election: true
I20260619 16:58:27.922371 22959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2e272e647cc4a6e8a69378a770f6821" candidate_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89b668bda708492c8caf7dcbaad7f693" is_pre_election: true
I20260619 16:58:27.922484 22819 raft_consensus.cc:2454] T e2e272e647cc4a6e8a69378a770f6821 P 35b7b94f6a984ce88fdf4c540fec5817 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b07ee522f96d4f4db5788b2bce6d8afc in term 0.
I20260619 16:58:27.922528 22959 raft_consensus.cc:2454] T e2e272e647cc4a6e8a69378a770f6821 P 89b668bda708492c8caf7dcbaad7f693 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b07ee522f96d4f4db5788b2bce6d8afc in term 0.
I20260619 16:58:27.922783 22613 leader_election.cc:302] T e2e272e647cc4a6e8a69378a770f6821 P b07ee522f96d4f4db5788b2bce6d8afc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35b7b94f6a984ce88fdf4c540fec5817, b07ee522f96d4f4db5788b2bce6d8afc; no voters:
I20260619 16:58:27.922930 23036 raft_consensus.cc:2781] T e2e272e647cc4a6e8a69378a770f6821 P b07ee522f96d4f4db5788b2bce6d8afc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:58:27.922996 23036 raft_consensus.cc:482] T e2e272e647cc4a6e8a69378a770f6821 P b07ee522f96d4f4db5788b2bce6d8afc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:58:27.923022 23036 raft_consensus.cc:3037] T e2e272e647cc4a6e8a69378a770f6821 P b07ee522f96d4f4db5788b2bce6d8afc [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:27.923713 23036 raft_consensus.cc:504] T e2e272e647cc4a6e8a69378a770f6821 P b07ee522f96d4f4db5788b2bce6d8afc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } } peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } }
I20260619 16:58:27.923846 23036 leader_election.cc:288] T e2e272e647cc4a6e8a69378a770f6821 P b07ee522f96d4f4db5788b2bce6d8afc [CANDIDATE]: Term 1 election: Requested vote from peers 35b7b94f6a984ce88fdf4c540fec5817 (127.20.231.130:33811), 89b668bda708492c8caf7dcbaad7f693 (127.20.231.131:40031)
I20260619 16:58:27.924036 22819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2e272e647cc4a6e8a69378a770f6821" candidate_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35b7b94f6a984ce88fdf4c540fec5817"
I20260619 16:58:27.924047 22959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2e272e647cc4a6e8a69378a770f6821" candidate_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89b668bda708492c8caf7dcbaad7f693"
I20260619 16:58:27.924122 22819 raft_consensus.cc:3037] T e2e272e647cc4a6e8a69378a770f6821 P 35b7b94f6a984ce88fdf4c540fec5817 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:27.924139 22959 raft_consensus.cc:3037] T e2e272e647cc4a6e8a69378a770f6821 P 89b668bda708492c8caf7dcbaad7f693 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:27.924880 22959 raft_consensus.cc:2454] T e2e272e647cc4a6e8a69378a770f6821 P 89b668bda708492c8caf7dcbaad7f693 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b07ee522f96d4f4db5788b2bce6d8afc in term 1.
I20260619 16:58:27.925063 22819 raft_consensus.cc:2454] T e2e272e647cc4a6e8a69378a770f6821 P 35b7b94f6a984ce88fdf4c540fec5817 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b07ee522f96d4f4db5788b2bce6d8afc in term 1.
I20260619 16:58:27.925079 22613 leader_election.cc:302] T e2e272e647cc4a6e8a69378a770f6821 P b07ee522f96d4f4db5788b2bce6d8afc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 89b668bda708492c8caf7dcbaad7f693, b07ee522f96d4f4db5788b2bce6d8afc; no voters:
I20260619 16:58:27.925170 23036 raft_consensus.cc:2781] T e2e272e647cc4a6e8a69378a770f6821 P b07ee522f96d4f4db5788b2bce6d8afc [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:27.925287 23036 raft_consensus.cc:686] T e2e272e647cc4a6e8a69378a770f6821 P b07ee522f96d4f4db5788b2bce6d8afc [term 1 LEADER]: Becoming Leader. State: Replica: b07ee522f96d4f4db5788b2bce6d8afc, State: Running, Role: LEADER
I20260619 16:58:27.925390 23036 consensus_queue.cc:237] T e2e272e647cc4a6e8a69378a770f6821 P b07ee522f96d4f4db5788b2bce6d8afc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } } peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } }
I20260619 16:58:27.926235 22521 catalog_manager.cc:5495] T e2e272e647cc4a6e8a69378a770f6821 P b07ee522f96d4f4db5788b2bce6d8afc reported cstate change: term changed from 0 to 1, leader changed from <none> to b07ee522f96d4f4db5788b2bce6d8afc (127.20.231.129). New cstate: current_term: 1 leader_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } health_report { overall_health: HEALTHY } } }
I20260619 16:58:27.934513 23025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:27.932057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36212 (local address 127.20.231.129:46431)
0619 16:58:27.932094 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:27.932097 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:27.932148 (+ 51us) server_negotiation.cc:407] Connection header received
0619 16:58:27.932229 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:27.932230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:27.932245 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:27.932303 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:58:27.932513 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.932990 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.933830 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:27.934006 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:27.934055 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:27.934244 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:58:27.934334 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:58:27.934391 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:27.934443 (+ 52us) server_negotiation.cc:299] Negotiation successful
0619 16:58:27.934460 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 16:58:27.936815 22959 raft_consensus.cc:1261] T e2e272e647cc4a6e8a69378a770f6821 P 89b668bda708492c8caf7dcbaad7f693 [term 1 FOLLOWER]: Refusing update from remote peer b07ee522f96d4f4db5788b2bce6d8afc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:27.936854 22819 raft_consensus.cc:1261] T e2e272e647cc4a6e8a69378a770f6821 P 35b7b94f6a984ce88fdf4c540fec5817 [term 1 FOLLOWER]: Refusing update from remote peer b07ee522f96d4f4db5788b2bce6d8afc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:27.937270 23036 consensus_queue.cc:1035] T e2e272e647cc4a6e8a69378a770f6821 P b07ee522f96d4f4db5788b2bce6d8afc [LEADER]: Connected to new peer: Peer: permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:27.937505 23039 consensus_queue.cc:1035] T e2e272e647cc4a6e8a69378a770f6821 P b07ee522f96d4f4db5788b2bce6d8afc [LEADER]: Connected to new peer: Peer: permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:27.940922 23043 mvcc.cc:205] Tried to move back new op lower bound from 7298614509305507840 to 7298614509262921728. Current Snapshot: MvccSnapshot[applied={T|T < 7298614509305507840 or (T in {7298614509305507840})}]
I20260619 16:58:27.943940 22522 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:58:27.944099 22522 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:58:27.945199 22522 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e9440f86afd64d71bad59055d60cb3a2: Sending DeleteTablet for 3 replicas of tablet e2e272e647cc4a6e8a69378a770f6821
I20260619 16:58:27.945683 22939 tablet_service.cc:1514] Processing DeleteTablet for tablet e2e272e647cc4a6e8a69378a770f6821 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:27 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:44620
I20260619 16:58:27.945672 22799 tablet_service.cc:1514] Processing DeleteTablet for tablet e2e272e647cc4a6e8a69378a770f6821 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:27 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:40194
I20260619 16:58:27.945992 22659 tablet_service.cc:1514] Processing DeleteTablet for tablet e2e272e647cc4a6e8a69378a770f6821 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:27 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:36204
I20260619 16:58:27.946095 22522 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54746:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:58:27.946376 22522 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:58:27.946502 23054 tablet_replica.cc:330] T e2e272e647cc4a6e8a69378a770f6821 P b07ee522f96d4f4db5788b2bce6d8afc: stopping tablet replica
I20260619 16:58:27.946610 23052 tablet_replica.cc:330] T e2e272e647cc4a6e8a69378a770f6821 P 89b668bda708492c8caf7dcbaad7f693: stopping tablet replica
I20260619 16:58:27.946739 23054 raft_consensus.cc:2229] T e2e272e647cc4a6e8a69378a770f6821 P b07ee522f96d4f4db5788b2bce6d8afc [term 1 LEADER]: Raft consensus shutting down.
I20260619 16:58:27.946866 23052 raft_consensus.cc:2229] T e2e272e647cc4a6e8a69378a770f6821 P 89b668bda708492c8caf7dcbaad7f693 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:58:27.947216 23052 raft_consensus.cc:2258] T e2e272e647cc4a6e8a69378a770f6821 P 89b668bda708492c8caf7dcbaad7f693 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:27.947633 23054 raft_consensus.cc:2258] T e2e272e647cc4a6e8a69378a770f6821 P b07ee522f96d4f4db5788b2bce6d8afc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:27.947731 23052 ts_tablet_manager.cc:1905] T e2e272e647cc4a6e8a69378a770f6821 P 89b668bda708492c8caf7dcbaad7f693: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:27.947832 22659 tablet_service.cc:1467] Processing CreateTablet for tablet 793c355dcbb746838427c3e791a4735a (DEFAULT_TABLE table=test-table [id=668338ad65f24d66881441c3e921347c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:27.947973 22659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 793c355dcbb746838427c3e791a4735a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:27.948187 23054 ts_tablet_manager.cc:1905] T e2e272e647cc4a6e8a69378a770f6821 P b07ee522f96d4f4db5788b2bce6d8afc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:27.949352 23031 tablet_bootstrap.cc:492] T 793c355dcbb746838427c3e791a4735a P b07ee522f96d4f4db5788b2bce6d8afc: Bootstrap starting.
I20260619 16:58:27.949443 22939 tablet_service.cc:1467] Processing CreateTablet for tablet 793c355dcbb746838427c3e791a4735a (DEFAULT_TABLE table=test-table [id=668338ad65f24d66881441c3e921347c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:27.949544 22939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 793c355dcbb746838427c3e791a4735a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:27.949702 23052 ts_tablet_manager.cc:1918] T e2e272e647cc4a6e8a69378a770f6821 P 89b668bda708492c8caf7dcbaad7f693: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:58:27.949800 23052 log.cc:1196] T e2e272e647cc4a6e8a69378a770f6821 P 89b668bda708492c8caf7dcbaad7f693: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-2/wal/wals/e2e272e647cc4a6e8a69378a770f6821
I20260619 16:58:27.949801 23031 tablet_bootstrap.cc:654] T 793c355dcbb746838427c3e791a4735a P b07ee522f96d4f4db5788b2bce6d8afc: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:27.950094 23052 ts_tablet_manager.cc:1939] T e2e272e647cc4a6e8a69378a770f6821 P 89b668bda708492c8caf7dcbaad7f693: Deleting consensus metadata
I20260619 16:58:27.950439 23053 tablet_replica.cc:330] T e2e272e647cc4a6e8a69378a770f6821 P 35b7b94f6a984ce88fdf4c540fec5817: stopping tablet replica
I20260619 16:58:27.950554 22799 tablet_service.cc:1467] Processing CreateTablet for tablet 793c355dcbb746838427c3e791a4735a (DEFAULT_TABLE table=test-table [id=668338ad65f24d66881441c3e921347c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:27.950685 22799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 793c355dcbb746838427c3e791a4735a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:27.950572 23053 raft_consensus.cc:2229] T e2e272e647cc4a6e8a69378a770f6821 P 35b7b94f6a984ce88fdf4c540fec5817 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:58:27.950980 23053 raft_consensus.cc:2258] T e2e272e647cc4a6e8a69378a770f6821 P 35b7b94f6a984ce88fdf4c540fec5817 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:27.951241 22509 catalog_manager.cc:4841] TS 89b668bda708492c8caf7dcbaad7f693 (127.20.231.131:40031): tablet e2e272e647cc4a6e8a69378a770f6821 (table test-table [id=65b9471487a84df5a5d984acff72e5f2]) successfully deleted
I20260619 16:58:27.951733 23053 ts_tablet_manager.cc:1905] T e2e272e647cc4a6e8a69378a770f6821 P 35b7b94f6a984ce88fdf4c540fec5817: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:27.952250 23054 ts_tablet_manager.cc:1918] T e2e272e647cc4a6e8a69378a770f6821 P b07ee522f96d4f4db5788b2bce6d8afc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:58:27.952382 23054 log.cc:1196] T e2e272e647cc4a6e8a69378a770f6821 P b07ee522f96d4f4db5788b2bce6d8afc: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-0/wal/wals/e2e272e647cc4a6e8a69378a770f6821
I20260619 16:58:27.952394 23029 tablet_bootstrap.cc:492] T 793c355dcbb746838427c3e791a4735a P 35b7b94f6a984ce88fdf4c540fec5817: Bootstrap starting.
I20260619 16:58:27.952725 23054 ts_tablet_manager.cc:1939] T e2e272e647cc4a6e8a69378a770f6821 P b07ee522f96d4f4db5788b2bce6d8afc: Deleting consensus metadata
I20260619 16:58:27.952782 23029 tablet_bootstrap.cc:654] T 793c355dcbb746838427c3e791a4735a P 35b7b94f6a984ce88fdf4c540fec5817: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:27.953239 22507 catalog_manager.cc:4841] TS b07ee522f96d4f4db5788b2bce6d8afc (127.20.231.129:46431): tablet e2e272e647cc4a6e8a69378a770f6821 (table test-table [id=65b9471487a84df5a5d984acff72e5f2]) successfully deleted
I20260619 16:58:27.953519 23053 ts_tablet_manager.cc:1918] T e2e272e647cc4a6e8a69378a770f6821 P 35b7b94f6a984ce88fdf4c540fec5817: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:58:27.953589 23053 log.cc:1196] T e2e272e647cc4a6e8a69378a770f6821 P 35b7b94f6a984ce88fdf4c540fec5817: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-1/wal/wals/e2e272e647cc4a6e8a69378a770f6821
I20260619 16:58:27.953856 23053 ts_tablet_manager.cc:1939] T e2e272e647cc4a6e8a69378a770f6821 P 35b7b94f6a984ce88fdf4c540fec5817: Deleting consensus metadata
I20260619 16:58:27.954010 23028 tablet_bootstrap.cc:492] T 793c355dcbb746838427c3e791a4735a P 89b668bda708492c8caf7dcbaad7f693: Bootstrap starting.
I20260619 16:58:27.954274 22509 catalog_manager.cc:4841] TS 35b7b94f6a984ce88fdf4c540fec5817 (127.20.231.130:33811): tablet e2e272e647cc4a6e8a69378a770f6821 (table test-table [id=65b9471487a84df5a5d984acff72e5f2]) successfully deleted
I20260619 16:58:27.954348 23031 tablet_bootstrap.cc:492] T 793c355dcbb746838427c3e791a4735a P b07ee522f96d4f4db5788b2bce6d8afc: No bootstrap required, opened a new log
I20260619 16:58:27.954387 23031 ts_tablet_manager.cc:1397] T 793c355dcbb746838427c3e791a4735a P b07ee522f96d4f4db5788b2bce6d8afc: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260619 16:58:27.954435 23028 tablet_bootstrap.cc:654] T 793c355dcbb746838427c3e791a4735a P 89b668bda708492c8caf7dcbaad7f693: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:27.954509 23031 raft_consensus.cc:348] T 793c355dcbb746838427c3e791a4735a P b07ee522f96d4f4db5788b2bce6d8afc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } } peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } }
I20260619 16:58:27.954571 23031 raft_consensus.cc:374] T 793c355dcbb746838427c3e791a4735a P b07ee522f96d4f4db5788b2bce6d8afc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:27.954584 23031 raft_consensus.cc:729] T 793c355dcbb746838427c3e791a4735a P b07ee522f96d4f4db5788b2bce6d8afc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b07ee522f96d4f4db5788b2bce6d8afc, State: Initialized, Role: FOLLOWER
I20260619 16:58:27.954658 23031 consensus_queue.cc:260] T 793c355dcbb746838427c3e791a4735a P b07ee522f96d4f4db5788b2bce6d8afc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } } peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } }
I20260619 16:58:27.954769 23031 ts_tablet_manager.cc:1428] T 793c355dcbb746838427c3e791a4735a P b07ee522f96d4f4db5788b2bce6d8afc: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 16:58:27.955080 23028 tablet_bootstrap.cc:492] T 793c355dcbb746838427c3e791a4735a P 89b668bda708492c8caf7dcbaad7f693: No bootstrap required, opened a new log
I20260619 16:58:27.955128 23028 ts_tablet_manager.cc:1397] T 793c355dcbb746838427c3e791a4735a P 89b668bda708492c8caf7dcbaad7f693: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:27.955195 23029 tablet_bootstrap.cc:492] T 793c355dcbb746838427c3e791a4735a P 35b7b94f6a984ce88fdf4c540fec5817: No bootstrap required, opened a new log
I20260619 16:58:27.955235 23029 ts_tablet_manager.cc:1397] T 793c355dcbb746838427c3e791a4735a P 35b7b94f6a984ce88fdf4c540fec5817: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 16:58:27.955282 23028 raft_consensus.cc:348] T 793c355dcbb746838427c3e791a4735a P 89b668bda708492c8caf7dcbaad7f693 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } } peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } }
I20260619 16:58:27.955353 23028 raft_consensus.cc:374] T 793c355dcbb746838427c3e791a4735a P 89b668bda708492c8caf7dcbaad7f693 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:27.955374 23028 raft_consensus.cc:729] T 793c355dcbb746838427c3e791a4735a P 89b668bda708492c8caf7dcbaad7f693 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89b668bda708492c8caf7dcbaad7f693, State: Initialized, Role: FOLLOWER
I20260619 16:58:27.955372 23029 raft_consensus.cc:348] T 793c355dcbb746838427c3e791a4735a P 35b7b94f6a984ce88fdf4c540fec5817 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } } peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } }
I20260619 16:58:27.955436 23029 raft_consensus.cc:374] T 793c355dcbb746838427c3e791a4735a P 35b7b94f6a984ce88fdf4c540fec5817 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:27.955425 23028 consensus_queue.cc:260] T 793c355dcbb746838427c3e791a4735a P 89b668bda708492c8caf7dcbaad7f693 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } } peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } }
I20260619 16:58:27.955457 23029 raft_consensus.cc:729] T 793c355dcbb746838427c3e791a4735a P 35b7b94f6a984ce88fdf4c540fec5817 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35b7b94f6a984ce88fdf4c540fec5817, State: Initialized, Role: FOLLOWER
I20260619 16:58:27.955505 23028 ts_tablet_manager.cc:1428] T 793c355dcbb746838427c3e791a4735a P 89b668bda708492c8caf7dcbaad7f693: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 16:58:27.955500 23029 consensus_queue.cc:260] T 793c355dcbb746838427c3e791a4735a P 35b7b94f6a984ce88fdf4c540fec5817 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } } peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } }
I20260619 16:58:27.955578 23029 ts_tablet_manager.cc:1428] T 793c355dcbb746838427c3e791a4735a P 35b7b94f6a984ce88fdf4c540fec5817: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:28.121356 23039 raft_consensus.cc:482] T 793c355dcbb746838427c3e791a4735a P b07ee522f96d4f4db5788b2bce6d8afc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:58:28.121451 23039 raft_consensus.cc:504] T 793c355dcbb746838427c3e791a4735a P b07ee522f96d4f4db5788b2bce6d8afc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } } peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } }
I20260619 16:58:28.121628 23039 leader_election.cc:288] T 793c355dcbb746838427c3e791a4735a P b07ee522f96d4f4db5788b2bce6d8afc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35b7b94f6a984ce88fdf4c540fec5817 (127.20.231.130:33811), 89b668bda708492c8caf7dcbaad7f693 (127.20.231.131:40031)
I20260619 16:58:28.121973 22819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "793c355dcbb746838427c3e791a4735a" candidate_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35b7b94f6a984ce88fdf4c540fec5817" is_pre_election: true
I20260619 16:58:28.122099 22819 raft_consensus.cc:2454] T 793c355dcbb746838427c3e791a4735a P 35b7b94f6a984ce88fdf4c540fec5817 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b07ee522f96d4f4db5788b2bce6d8afc in term 0.
I20260619 16:58:28.122058 22959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "793c355dcbb746838427c3e791a4735a" candidate_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89b668bda708492c8caf7dcbaad7f693" is_pre_election: true
I20260619 16:58:28.122208 22959 raft_consensus.cc:2454] T 793c355dcbb746838427c3e791a4735a P 89b668bda708492c8caf7dcbaad7f693 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b07ee522f96d4f4db5788b2bce6d8afc in term 0.
I20260619 16:58:28.122294 22613 leader_election.cc:302] T 793c355dcbb746838427c3e791a4735a P b07ee522f96d4f4db5788b2bce6d8afc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35b7b94f6a984ce88fdf4c540fec5817, b07ee522f96d4f4db5788b2bce6d8afc; no voters:
I20260619 16:58:28.122423 23039 raft_consensus.cc:2781] T 793c355dcbb746838427c3e791a4735a P b07ee522f96d4f4db5788b2bce6d8afc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:58:28.122478 23039 raft_consensus.cc:482] T 793c355dcbb746838427c3e791a4735a P b07ee522f96d4f4db5788b2bce6d8afc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:58:28.122494 23039 raft_consensus.cc:3037] T 793c355dcbb746838427c3e791a4735a P b07ee522f96d4f4db5788b2bce6d8afc [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:28.123358 23039 raft_consensus.cc:504] T 793c355dcbb746838427c3e791a4735a P b07ee522f96d4f4db5788b2bce6d8afc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } } peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } }
I20260619 16:58:28.123546 23039 leader_election.cc:288] T 793c355dcbb746838427c3e791a4735a P b07ee522f96d4f4db5788b2bce6d8afc [CANDIDATE]: Term 1 election: Requested vote from peers 35b7b94f6a984ce88fdf4c540fec5817 (127.20.231.130:33811), 89b668bda708492c8caf7dcbaad7f693 (127.20.231.131:40031)
I20260619 16:58:28.123723 22819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "793c355dcbb746838427c3e791a4735a" candidate_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35b7b94f6a984ce88fdf4c540fec5817"
I20260619 16:58:28.123807 22819 raft_consensus.cc:3037] T 793c355dcbb746838427c3e791a4735a P 35b7b94f6a984ce88fdf4c540fec5817 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:28.123814 22959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "793c355dcbb746838427c3e791a4735a" candidate_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89b668bda708492c8caf7dcbaad7f693"
I20260619 16:58:28.123951 22959 raft_consensus.cc:3037] T 793c355dcbb746838427c3e791a4735a P 89b668bda708492c8caf7dcbaad7f693 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:28.124519 22819 raft_consensus.cc:2454] T 793c355dcbb746838427c3e791a4735a P 35b7b94f6a984ce88fdf4c540fec5817 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b07ee522f96d4f4db5788b2bce6d8afc in term 1.
I20260619 16:58:28.124583 22959 raft_consensus.cc:2454] T 793c355dcbb746838427c3e791a4735a P 89b668bda708492c8caf7dcbaad7f693 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b07ee522f96d4f4db5788b2bce6d8afc in term 1.
I20260619 16:58:28.124696 22613 leader_election.cc:302] T 793c355dcbb746838427c3e791a4735a P b07ee522f96d4f4db5788b2bce6d8afc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35b7b94f6a984ce88fdf4c540fec5817, b07ee522f96d4f4db5788b2bce6d8afc; no voters:
I20260619 16:58:28.124827 23039 raft_consensus.cc:2781] T 793c355dcbb746838427c3e791a4735a P b07ee522f96d4f4db5788b2bce6d8afc [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:28.124936 23039 raft_consensus.cc:686] T 793c355dcbb746838427c3e791a4735a P b07ee522f96d4f4db5788b2bce6d8afc [term 1 LEADER]: Becoming Leader. State: Replica: b07ee522f96d4f4db5788b2bce6d8afc, State: Running, Role: LEADER
I20260619 16:58:28.125003 23039 consensus_queue.cc:237] T 793c355dcbb746838427c3e791a4735a P b07ee522f96d4f4db5788b2bce6d8afc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } } peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } }
I20260619 16:58:28.125630 22522 catalog_manager.cc:5495] T 793c355dcbb746838427c3e791a4735a P b07ee522f96d4f4db5788b2bce6d8afc reported cstate change: term changed from 0 to 1, leader changed from <none> to b07ee522f96d4f4db5788b2bce6d8afc (127.20.231.129). New cstate: current_term: 1 leader_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } health_report { overall_health: UNKNOWN } } }
Jun 19 16:58:28 dist-test-slave-vnbj krb5kdc[22477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888307, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:28.163484 22583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:28.155650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54750 (local address 127.20.231.190:33825)
0619 16:58:28.155799 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:28.155801 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:28.155860 (+ 59us) server_negotiation.cc:407] Connection header received
0619 16:58:28.155940 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:28.155940 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:28.155950 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:28.155991 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:28.156464 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:28.157049 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:28.157813 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:28.157992 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:28.158103 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:28.160754 (+ 2651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:28.160755 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:28.160755 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:28.160762 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:28.162353 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:28.162707 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:28.162707 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:28.162708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:28.162755 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:28.163046 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:28.163047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:28.163047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:28.163165 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:28.163217 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:28.163351 (+ 134us) server_negotiation.cc:299] Negotiation successful
0619 16:58:28.163417 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260619 16:58:28.163642 23064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:28.155602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33825 (local address 127.0.0.1:54750)
0619 16:58:28.155792 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:28.155804 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:28.155882 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:28.156010 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:28.156010 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:28.156330 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:28.156377 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:28.156379 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:28.157144 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:28.157145 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:28.157727 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:28.157729 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:28.158119 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:28.158120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:28.158173 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:28.158746 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:28.158751 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:28.160654 (+ 1903us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:28.162430 (+ 1776us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:28.162437 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:28.162439 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:28.162641 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:28.162934 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:28.162934 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:28.162935 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:28.162990 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:28.163229 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:28.163235 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:28.163280 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 16:58:28.163337 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 16:58:28.163480 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":54,"thread_start_us":108,"threads_started":1}
I20260619 16:58:28.165904 22522 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:54750:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 16:58:28.166085 22522 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:58:28.167490 22799 tablet_service.cc:1467] Processing CreateTablet for tablet 4ff249c6726c40e99c9f5a1a8619e68a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0fa40d47c16d4e1f9096f015963678e2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 16:58:28.167512 22939 tablet_service.cc:1467] Processing CreateTablet for tablet 4ff249c6726c40e99c9f5a1a8619e68a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0fa40d47c16d4e1f9096f015963678e2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 16:58:28.167512 22659 tablet_service.cc:1467] Processing CreateTablet for tablet 4ff249c6726c40e99c9f5a1a8619e68a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0fa40d47c16d4e1f9096f015963678e2]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 16:58:28.167656 22799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ff249c6726c40e99c9f5a1a8619e68a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:28.167656 22939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ff249c6726c40e99c9f5a1a8619e68a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:28.167672 22659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ff249c6726c40e99c9f5a1a8619e68a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:28.168838 23031 tablet_bootstrap.cc:492] T 4ff249c6726c40e99c9f5a1a8619e68a P b07ee522f96d4f4db5788b2bce6d8afc: Bootstrap starting.
I20260619 16:58:28.169080 23028 tablet_bootstrap.cc:492] T 4ff249c6726c40e99c9f5a1a8619e68a P 89b668bda708492c8caf7dcbaad7f693: Bootstrap starting.
I20260619 16:58:28.169106 23029 tablet_bootstrap.cc:492] T 4ff249c6726c40e99c9f5a1a8619e68a P 35b7b94f6a984ce88fdf4c540fec5817: Bootstrap starting.
I20260619 16:58:28.169178 23031 tablet_bootstrap.cc:654] T 4ff249c6726c40e99c9f5a1a8619e68a P b07ee522f96d4f4db5788b2bce6d8afc: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:28.169509 23028 tablet_bootstrap.cc:654] T 4ff249c6726c40e99c9f5a1a8619e68a P 89b668bda708492c8caf7dcbaad7f693: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:28.169533 23029 tablet_bootstrap.cc:654] T 4ff249c6726c40e99c9f5a1a8619e68a P 35b7b94f6a984ce88fdf4c540fec5817: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:28.169773 23031 tablet_bootstrap.cc:492] T 4ff249c6726c40e99c9f5a1a8619e68a P b07ee522f96d4f4db5788b2bce6d8afc: No bootstrap required, opened a new log
I20260619 16:58:28.169833 23031 ts_tablet_manager.cc:1397] T 4ff249c6726c40e99c9f5a1a8619e68a P b07ee522f96d4f4db5788b2bce6d8afc: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 16:58:28.169945 23031 raft_consensus.cc:348] T 4ff249c6726c40e99c9f5a1a8619e68a P b07ee522f96d4f4db5788b2bce6d8afc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } } peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } }
I20260619 16:58:28.170022 23031 raft_consensus.cc:374] T 4ff249c6726c40e99c9f5a1a8619e68a P b07ee522f96d4f4db5788b2bce6d8afc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:28.170047 23031 raft_consensus.cc:729] T 4ff249c6726c40e99c9f5a1a8619e68a P b07ee522f96d4f4db5788b2bce6d8afc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b07ee522f96d4f4db5788b2bce6d8afc, State: Initialized, Role: FOLLOWER
I20260619 16:58:28.170096 23031 consensus_queue.cc:260] T 4ff249c6726c40e99c9f5a1a8619e68a P b07ee522f96d4f4db5788b2bce6d8afc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } } peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } }
I20260619 16:58:28.170130 23029 tablet_bootstrap.cc:492] T 4ff249c6726c40e99c9f5a1a8619e68a P 35b7b94f6a984ce88fdf4c540fec5817: No bootstrap required, opened a new log
I20260619 16:58:28.170174 23029 ts_tablet_manager.cc:1397] T 4ff249c6726c40e99c9f5a1a8619e68a P 35b7b94f6a984ce88fdf4c540fec5817: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:28.170185 23031 ts_tablet_manager.cc:1428] T 4ff249c6726c40e99c9f5a1a8619e68a P b07ee522f96d4f4db5788b2bce6d8afc: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:28.170198 23039 tablet_replica.cc:439] T 4ff249c6726c40e99c9f5a1a8619e68a P b07ee522f96d4f4db5788b2bce6d8afc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } } peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } } }
I20260619 16:58:28.170276 23028 tablet_bootstrap.cc:492] T 4ff249c6726c40e99c9f5a1a8619e68a P 89b668bda708492c8caf7dcbaad7f693: No bootstrap required, opened a new log
I20260619 16:58:28.170264 23039 tablet_replica.cc:442] T 4ff249c6726c40e99c9f5a1a8619e68a P b07ee522f96d4f4db5788b2bce6d8afc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:58:28.170321 23028 ts_tablet_manager.cc:1397] T 4ff249c6726c40e99c9f5a1a8619e68a P 89b668bda708492c8caf7dcbaad7f693: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:28.170320 23029 raft_consensus.cc:348] T 4ff249c6726c40e99c9f5a1a8619e68a P 35b7b94f6a984ce88fdf4c540fec5817 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } } peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } }
I20260619 16:58:28.170405 23029 raft_consensus.cc:374] T 4ff249c6726c40e99c9f5a1a8619e68a P 35b7b94f6a984ce88fdf4c540fec5817 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:28.170424 23029 raft_consensus.cc:729] T 4ff249c6726c40e99c9f5a1a8619e68a P 35b7b94f6a984ce88fdf4c540fec5817 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35b7b94f6a984ce88fdf4c540fec5817, State: Initialized, Role: FOLLOWER
I20260619 16:58:28.170472 23029 consensus_queue.cc:260] T 4ff249c6726c40e99c9f5a1a8619e68a P 35b7b94f6a984ce88fdf4c540fec5817 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } } peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } }
I20260619 16:58:28.170460 23028 raft_consensus.cc:348] T 4ff249c6726c40e99c9f5a1a8619e68a P 89b668bda708492c8caf7dcbaad7f693 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } } peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } }
I20260619 16:58:28.170539 23028 raft_consensus.cc:374] T 4ff249c6726c40e99c9f5a1a8619e68a P 89b668bda708492c8caf7dcbaad7f693 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:28.170567 23028 raft_consensus.cc:729] T 4ff249c6726c40e99c9f5a1a8619e68a P 89b668bda708492c8caf7dcbaad7f693 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89b668bda708492c8caf7dcbaad7f693, State: Initialized, Role: FOLLOWER
I20260619 16:58:28.170559 23029 ts_tablet_manager.cc:1428] T 4ff249c6726c40e99c9f5a1a8619e68a P 35b7b94f6a984ce88fdf4c540fec5817: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 16:58:28.170612 23028 consensus_queue.cc:260] T 4ff249c6726c40e99c9f5a1a8619e68a P 89b668bda708492c8caf7dcbaad7f693 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } } peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } }
I20260619 16:58:28.170710 23028 ts_tablet_manager.cc:1428] T 4ff249c6726c40e99c9f5a1a8619e68a P 89b668bda708492c8caf7dcbaad7f693: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 16:58:28.170717 23035 tablet_replica.cc:439] T 4ff249c6726c40e99c9f5a1a8619e68a P 35b7b94f6a984ce88fdf4c540fec5817: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } } peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } } }
I20260619 16:58:28.170806 23035 tablet_replica.cc:442] T 4ff249c6726c40e99c9f5a1a8619e68a P 35b7b94f6a984ce88fdf4c540fec5817: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:58:28.170814 23034 tablet_replica.cc:439] T 4ff249c6726c40e99c9f5a1a8619e68a P 89b668bda708492c8caf7dcbaad7f693: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } } peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } } }
I20260619 16:58:28.170953 23034 tablet_replica.cc:442] T 4ff249c6726c40e99c9f5a1a8619e68a P 89b668bda708492c8caf7dcbaad7f693: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:58:28.230111 23035 raft_consensus.cc:482] T 4ff249c6726c40e99c9f5a1a8619e68a P 35b7b94f6a984ce88fdf4c540fec5817 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:58:28.230216 23035 raft_consensus.cc:504] T 4ff249c6726c40e99c9f5a1a8619e68a P 35b7b94f6a984ce88fdf4c540fec5817 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } } peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } }
I20260619 16:58:28.230612 23035 leader_election.cc:288] T 4ff249c6726c40e99c9f5a1a8619e68a P 35b7b94f6a984ce88fdf4c540fec5817 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b07ee522f96d4f4db5788b2bce6d8afc (127.20.231.129:46431), 89b668bda708492c8caf7dcbaad7f693 (127.20.231.131:40031)
I20260619 16:58:28.234331 23025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:28.231000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:50935 (local address 127.20.231.129:46431)
0619 16:58:28.231051 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:28.231054 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:28.231227 (+ 173us) server_negotiation.cc:407] Connection header received
0619 16:58:28.231283 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:28.231284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:28.231297 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:28.231358 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:28.231544 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:28.232260 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:28.233431 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:28.233906 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:28.234000 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:28.234034 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:28.234220 (+ 186us) server_negotiation.cc:299] Negotiation successful
0619 16:58:28.234264 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 16:58:28.234330 23027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:28.230923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:51967 (local address 127.20.231.131:40031)
0619 16:58:28.231005 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:28.231008 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:28.231134 (+ 126us) server_negotiation.cc:407] Connection header received
0619 16:58:28.231208 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:28.231209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:28.231220 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:28.231278 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:28.231427 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:28.232234 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:28.233495 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:28.233851 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:28.233910 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:28.233917 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:28.234193 (+ 276us) server_negotiation.cc:299] Negotiation successful
0619 16:58:28.234244 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 16:58:28.234364 23068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:28.230819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:46431 (local address 127.20.231.130:50935)
0619 16:58:28.231032 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:28.231093 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:28.231215 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:28.231380 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:28.231381 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:28.231383 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:28.231468 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:28.231469 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:28.232376 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:28.232377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:28.233323 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:28.233325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:28.234003 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:28.234004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:28.234105 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:28.234110 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 16:58:28.234205 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 16:58:28.234257 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":87,"thread_start_us":156,"threads_started":1}
I20260619 16:58:28.234620 22679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff249c6726c40e99c9f5a1a8619e68a" candidate_uuid: "35b7b94f6a984ce88fdf4c540fec5817" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" is_pre_election: true
I20260619 16:58:28.234748 22679 raft_consensus.cc:2454] T 4ff249c6726c40e99c9f5a1a8619e68a P b07ee522f96d4f4db5788b2bce6d8afc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35b7b94f6a984ce88fdf4c540fec5817 in term 0.
I20260619 16:58:28.234376 23069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:28.230818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:40031 (local address 127.20.231.130:51967)
0619 16:58:28.231036 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:28.231046 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:28.231121 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:28.231281 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:28.231282 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:28.231284 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:28.231365 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:28.231367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:28.232345 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:28.232346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:28.233358 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:28.233360 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:28.233970 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:28.233971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:28.234047 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:28.234053 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 16:58:28.234186 (+ 133us) client_negotiation.cc:240] Negotiation successful
0619 16:58:28.234242 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":8,"thread_start_us":106,"threads_started":1}
I20260619 16:58:28.234985 22751 leader_election.cc:302] T 4ff249c6726c40e99c9f5a1a8619e68a P 35b7b94f6a984ce88fdf4c540fec5817 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35b7b94f6a984ce88fdf4c540fec5817, b07ee522f96d4f4db5788b2bce6d8afc; no voters:
I20260619 16:58:28.235157 23035 raft_consensus.cc:2781] T 4ff249c6726c40e99c9f5a1a8619e68a P 35b7b94f6a984ce88fdf4c540fec5817 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:58:28.235212 23035 raft_consensus.cc:482] T 4ff249c6726c40e99c9f5a1a8619e68a P 35b7b94f6a984ce88fdf4c540fec5817 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:58:28.235244 23035 raft_consensus.cc:3037] T 4ff249c6726c40e99c9f5a1a8619e68a P 35b7b94f6a984ce88fdf4c540fec5817 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:28.235301 22959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff249c6726c40e99c9f5a1a8619e68a" candidate_uuid: "35b7b94f6a984ce88fdf4c540fec5817" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89b668bda708492c8caf7dcbaad7f693" is_pre_election: true
I20260619 16:58:28.235410 22959 raft_consensus.cc:2454] T 4ff249c6726c40e99c9f5a1a8619e68a P 89b668bda708492c8caf7dcbaad7f693 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35b7b94f6a984ce88fdf4c540fec5817 in term 0.
I20260619 16:58:28.236155 23035 raft_consensus.cc:504] T 4ff249c6726c40e99c9f5a1a8619e68a P 35b7b94f6a984ce88fdf4c540fec5817 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } } peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } }
I20260619 16:58:28.236334 23035 leader_election.cc:288] T 4ff249c6726c40e99c9f5a1a8619e68a P 35b7b94f6a984ce88fdf4c540fec5817 [CANDIDATE]: Term 1 election: Requested vote from peers b07ee522f96d4f4db5788b2bce6d8afc (127.20.231.129:46431), 89b668bda708492c8caf7dcbaad7f693 (127.20.231.131:40031)
I20260619 16:58:28.236500 22959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff249c6726c40e99c9f5a1a8619e68a" candidate_uuid: "35b7b94f6a984ce88fdf4c540fec5817" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "89b668bda708492c8caf7dcbaad7f693"
I20260619 16:58:28.236521 22679 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff249c6726c40e99c9f5a1a8619e68a" candidate_uuid: "35b7b94f6a984ce88fdf4c540fec5817" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b07ee522f96d4f4db5788b2bce6d8afc"
I20260619 16:58:28.236598 22959 raft_consensus.cc:3037] T 4ff249c6726c40e99c9f5a1a8619e68a P 89b668bda708492c8caf7dcbaad7f693 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:28.236608 22679 raft_consensus.cc:3037] T 4ff249c6726c40e99c9f5a1a8619e68a P b07ee522f96d4f4db5788b2bce6d8afc [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:28.237465 22959 raft_consensus.cc:2454] T 4ff249c6726c40e99c9f5a1a8619e68a P 89b668bda708492c8caf7dcbaad7f693 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35b7b94f6a984ce88fdf4c540fec5817 in term 1.
I20260619 16:58:28.237466 22679 raft_consensus.cc:2454] T 4ff249c6726c40e99c9f5a1a8619e68a P b07ee522f96d4f4db5788b2bce6d8afc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35b7b94f6a984ce88fdf4c540fec5817 in term 1.
I20260619 16:58:28.237762 22751 leader_election.cc:302] T 4ff249c6726c40e99c9f5a1a8619e68a P 35b7b94f6a984ce88fdf4c540fec5817 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35b7b94f6a984ce88fdf4c540fec5817, b07ee522f96d4f4db5788b2bce6d8afc; no voters:
I20260619 16:58:28.237949 23035 raft_consensus.cc:2781] T 4ff249c6726c40e99c9f5a1a8619e68a P 35b7b94f6a984ce88fdf4c540fec5817 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:28.238114 23035 raft_consensus.cc:686] T 4ff249c6726c40e99c9f5a1a8619e68a P 35b7b94f6a984ce88fdf4c540fec5817 [term 1 LEADER]: Becoming Leader. State: Replica: 35b7b94f6a984ce88fdf4c540fec5817, State: Running, Role: LEADER
I20260619 16:58:28.238211 23035 consensus_queue.cc:237] T 4ff249c6726c40e99c9f5a1a8619e68a P 35b7b94f6a984ce88fdf4c540fec5817 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } } peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } }
I20260619 16:58:28.238638 23070 tablet_replica.cc:439] T 4ff249c6726c40e99c9f5a1a8619e68a P 35b7b94f6a984ce88fdf4c540fec5817: TxnStatusTablet state changed. Reason: New leader 35b7b94f6a984ce88fdf4c540fec5817. Latest consensus state: current_term: 1 leader_uuid: "35b7b94f6a984ce88fdf4c540fec5817" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } } peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } } }
I20260619 16:58:28.238744 23070 tablet_replica.cc:442] T 4ff249c6726c40e99c9f5a1a8619e68a P 35b7b94f6a984ce88fdf4c540fec5817: This TxnStatusTablet replica's current role is: LEADER
I20260619 16:58:28.239056 23072 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 16:58:28.239213 23072 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 16:58:28.239120 22522 catalog_manager.cc:5495] T 4ff249c6726c40e99c9f5a1a8619e68a P 35b7b94f6a984ce88fdf4c540fec5817 reported cstate change: term changed from 0 to 1, leader changed from <none> to 35b7b94f6a984ce88fdf4c540fec5817 (127.20.231.130). New cstate: current_term: 1 leader_uuid: "35b7b94f6a984ce88fdf4c540fec5817" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } health_report { overall_health: UNKNOWN } } }
I20260619 16:58:28.248078 23064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:28.245446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:33811 (local address 127.0.0.1:40204)
0619 16:58:28.245521 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:28.245533 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:28.245582 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:28.245753 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:28.245754 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:28.245755 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 16:58:28.245805 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:28.245807 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:28.246486 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:28.246487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:28.247159 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:28.247161 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:28.247546 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:28.247547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:28.247609 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:28.247629 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 16:58:28.247900 (+ 271us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 16:58:28.247911 (+ 11us) client_negotiation.cc:769] Sending connection context
0619 16:58:28.247961 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 16:58:28.248011 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 16:58:28.248065 23024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:28.245524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40204 (local address 127.20.231.130:33811)
0619 16:58:28.245571 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:28.245573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:28.245583 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:58:28.245666 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:28.245667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:28.245685 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:28.245749 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:58:28.245885 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:28.246399 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:28.247258 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:28.247472 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:28.247548 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:28.247726 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:58:28.247833 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:58:28.247916 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:28.247963 (+ 47us) server_negotiation.cc:299] Negotiation successful
0619 16:58:28.247987 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 16:58:28.249482 22959 raft_consensus.cc:1261] T 4ff249c6726c40e99c9f5a1a8619e68a P 89b668bda708492c8caf7dcbaad7f693 [term 1 FOLLOWER]: Refusing update from remote peer 35b7b94f6a984ce88fdf4c540fec5817: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:28.249581 22679 raft_consensus.cc:1261] T 4ff249c6726c40e99c9f5a1a8619e68a P b07ee522f96d4f4db5788b2bce6d8afc [term 1 FOLLOWER]: Refusing update from remote peer 35b7b94f6a984ce88fdf4c540fec5817: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:28.249783 23070 consensus_queue.cc:1035] T 4ff249c6726c40e99c9f5a1a8619e68a P 35b7b94f6a984ce88fdf4c540fec5817 [LEADER]: Connected to new peer: Peer: permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:28.249889 23035 consensus_queue.cc:1035] T 4ff249c6726c40e99c9f5a1a8619e68a P 35b7b94f6a984ce88fdf4c540fec5817 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:28.250662 23039 tablet_replica.cc:439] T 4ff249c6726c40e99c9f5a1a8619e68a P b07ee522f96d4f4db5788b2bce6d8afc: TxnStatusTablet state changed. Reason: New leader 35b7b94f6a984ce88fdf4c540fec5817. Latest consensus state: current_term: 1 leader_uuid: "35b7b94f6a984ce88fdf4c540fec5817" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } } peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } } }
I20260619 16:58:28.250772 23039 tablet_replica.cc:442] T 4ff249c6726c40e99c9f5a1a8619e68a P b07ee522f96d4f4db5788b2bce6d8afc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:58:28.250780 23034 tablet_replica.cc:439] T 4ff249c6726c40e99c9f5a1a8619e68a P 89b668bda708492c8caf7dcbaad7f693: TxnStatusTablet state changed. Reason: New leader 35b7b94f6a984ce88fdf4c540fec5817. Latest consensus state: current_term: 1 leader_uuid: "35b7b94f6a984ce88fdf4c540fec5817" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } } peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } } }
I20260619 16:58:28.250897 23034 tablet_replica.cc:442] T 4ff249c6726c40e99c9f5a1a8619e68a P 89b668bda708492c8caf7dcbaad7f693: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:58:28.251678 23070 tablet_replica.cc:439] T 4ff249c6726c40e99c9f5a1a8619e68a P 35b7b94f6a984ce88fdf4c540fec5817: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "35b7b94f6a984ce88fdf4c540fec5817" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } } peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } } }
I20260619 16:58:28.251772 23070 tablet_replica.cc:442] T 4ff249c6726c40e99c9f5a1a8619e68a P 35b7b94f6a984ce88fdf4c540fec5817: This TxnStatusTablet replica's current role is: LEADER
I20260619 16:58:28.252077 23044 mvcc.cc:205] Tried to move back new op lower bound from 7298614510587002880 to 7298614510544482304. Current Snapshot: MvccSnapshot[applied={T|T < 7298614510587002880 or (T in {7298614510587002880})}]
I20260619 16:58:28.252431 23039 tablet_replica.cc:439] T 4ff249c6726c40e99c9f5a1a8619e68a P b07ee522f96d4f4db5788b2bce6d8afc: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "35b7b94f6a984ce88fdf4c540fec5817" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } } peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } } }
I20260619 16:58:28.252525 23039 tablet_replica.cc:442] T 4ff249c6726c40e99c9f5a1a8619e68a P b07ee522f96d4f4db5788b2bce6d8afc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:58:28.253244 23035 tablet_replica.cc:439] T 4ff249c6726c40e99c9f5a1a8619e68a P 35b7b94f6a984ce88fdf4c540fec5817: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "35b7b94f6a984ce88fdf4c540fec5817" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } } peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } } }
I20260619 16:58:28.253331 23035 tablet_replica.cc:442] T 4ff249c6726c40e99c9f5a1a8619e68a P 35b7b94f6a984ce88fdf4c540fec5817: This TxnStatusTablet replica's current role is: LEADER
I20260619 16:58:28.253592 23034 tablet_replica.cc:439] T 4ff249c6726c40e99c9f5a1a8619e68a P 89b668bda708492c8caf7dcbaad7f693: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "35b7b94f6a984ce88fdf4c540fec5817" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b07ee522f96d4f4db5788b2bce6d8afc" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46431 } } peers { permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 } } peers { permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 } } }
I20260619 16:58:28.253690 23034 tablet_replica.cc:442] T 4ff249c6726c40e99c9f5a1a8619e68a P 89b668bda708492c8caf7dcbaad7f693: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:58:28.258325 23024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:28.255370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:43555 (local address 127.20.231.130:33811)
0619 16:58:28.255413 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:28.255415 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:28.255538 (+ 123us) server_negotiation.cc:407] Connection header received
0619 16:58:28.255583 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:28.255584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:28.255595 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:28.255653 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:58:28.255755 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:28.256611 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:28.257520 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:28.257859 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:28.257936 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:28.258083 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:58:28.258119 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:58:28.258176 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:28.258222 (+ 46us) server_negotiation.cc:299] Negotiation successful
0619 16:58:28.258243 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 16:58:28.258379 23080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:28.255302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:33811 (local address 127.20.231.129:43555)
0619 16:58:28.255457 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:28.255468 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:28.255528 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:28.255663 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:28.255663 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:28.255665 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 16:58:28.255708 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:28.255709 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:28.256726 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:28.256727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:28.257454 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:28.257457 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:28.257939 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:28.257940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:28.257987 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:28.258002 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 16:58:28.258183 (+ 181us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 16:58:28.258184 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 16:58:28.258219 (+ 35us) client_negotiation.cc:240] Negotiation successful
0619 16:58:28.258255 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":85,"threads_started":1}
I20260619 16:58:28.260903 22959 raft_consensus.cc:1261] T 793c355dcbb746838427c3e791a4735a P 89b668bda708492c8caf7dcbaad7f693 [term 1 FOLLOWER]: Refusing update from remote peer b07ee522f96d4f4db5788b2bce6d8afc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:28.260950 22819 raft_consensus.cc:1261] T 793c355dcbb746838427c3e791a4735a P 35b7b94f6a984ce88fdf4c540fec5817 [term 1 FOLLOWER]: Refusing update from remote peer b07ee522f96d4f4db5788b2bce6d8afc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:28.261142 23047 consensus_queue.cc:1035] T 793c355dcbb746838427c3e791a4735a P b07ee522f96d4f4db5788b2bce6d8afc [LEADER]: Connected to new peer: Peer: permanent_uuid: "35b7b94f6a984ce88fdf4c540fec5817" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 33811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:28.261268 23039 consensus_queue.cc:1035] T 793c355dcbb746838427c3e791a4735a P b07ee522f96d4f4db5788b2bce6d8afc [LEADER]: Connected to new peer: Peer: permanent_uuid: "89b668bda708492c8caf7dcbaad7f693" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40031 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:28.262955 23042 mvcc.cc:205] Tried to move back new op lower bound from 7298614510635229184 to 7298614510080344064. Current Snapshot: MvccSnapshot[applied={T|T < 7298614510635229184}]
I20260619 16:58:28.269861 23025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:28.266901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:48795 (local address 127.20.231.129:46431)
0619 16:58:28.266944 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:28.266946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:28.266957 (+ 11us) server_negotiation.cc:407] Connection header received
0619 16:58:28.266982 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:28.266983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:28.266994 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:28.267062 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:58:28.267200 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:28.268021 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:28.269014 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:28.269405 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:28.269487 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:28.269621 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:58:28.269658 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:58:28.269715 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:28.269758 (+ 43us) server_negotiation.cc:299] Negotiation successful
0619 16:58:28.269780 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 16:58:28.269884 23084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:28.266673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:46431 (local address 127.20.231.130:48795)
0619 16:58:28.266854 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:28.266883 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:28.266936 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:28.267062 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:28.267063 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:28.267064 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 16:58:28.267134 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:28.267135 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:28.268104 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:28.268105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:28.268932 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:28.268934 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:28.269487 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:28.269488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:28.269533 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:28.269546 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 16:58:28.269718 (+ 172us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 16:58:28.269720 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 16:58:28.269753 (+ 33us) client_negotiation.cc:240] Negotiation successful
0619 16:58:28.269785 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":92,"threads_started":1}
I20260619 16:58:28.282231 22521 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:58:28.282346 22521 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54746:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:58:28.283063 22521 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e9440f86afd64d71bad59055d60cb3a2: Sending DeleteTablet for 3 replicas of tablet 793c355dcbb746838427c3e791a4735a
I20260619 16:58:28.283358 22659 tablet_service.cc:1514] Processing DeleteTablet for tablet 793c355dcbb746838427c3e791a4735a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:28 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:36204
I20260619 16:58:28.283403 22939 tablet_service.cc:1514] Processing DeleteTablet for tablet 793c355dcbb746838427c3e791a4735a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:28 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:44620
I20260619 16:58:28.283478 23054 tablet_replica.cc:330] T 793c355dcbb746838427c3e791a4735a P b07ee522f96d4f4db5788b2bce6d8afc: stopping tablet replica
I20260619 16:58:28.283501 23052 tablet_replica.cc:330] T 793c355dcbb746838427c3e791a4735a P 89b668bda708492c8caf7dcbaad7f693: stopping tablet replica
I20260619 16:58:28.283546 23054 raft_consensus.cc:2229] T 793c355dcbb746838427c3e791a4735a P b07ee522f96d4f4db5788b2bce6d8afc [term 1 LEADER]: Raft consensus shutting down.
I20260619 16:58:28.283555 23052 raft_consensus.cc:2229] T 793c355dcbb746838427c3e791a4735a P 89b668bda708492c8caf7dcbaad7f693 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:58:28.283599 23052 raft_consensus.cc:2258] T 793c355dcbb746838427c3e791a4735a P 89b668bda708492c8caf7dcbaad7f693 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:28.283591 22799 tablet_service.cc:1514] Processing DeleteTablet for tablet 793c355dcbb746838427c3e791a4735a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:28 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:40194
I20260619 16:58:28.283615 23054 raft_consensus.cc:2258] T 793c355dcbb746838427c3e791a4735a P b07ee522f96d4f4db5788b2bce6d8afc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:28.283922 23053 tablet_replica.cc:330] T 793c355dcbb746838427c3e791a4735a P 35b7b94f6a984ce88fdf4c540fec5817: stopping tablet replica
I20260619 16:58:28.283982 23053 raft_consensus.cc:2229] T 793c355dcbb746838427c3e791a4735a P 35b7b94f6a984ce88fdf4c540fec5817 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:58:28.284018 23053 raft_consensus.cc:2258] T 793c355dcbb746838427c3e791a4735a P 35b7b94f6a984ce88fdf4c540fec5817 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:28.284005 23054 ts_tablet_manager.cc:1905] T 793c355dcbb746838427c3e791a4735a P b07ee522f96d4f4db5788b2bce6d8afc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:28.284015 23052 ts_tablet_manager.cc:1905] T 793c355dcbb746838427c3e791a4735a P 89b668bda708492c8caf7dcbaad7f693: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:28.284595 23053 ts_tablet_manager.cc:1905] T 793c355dcbb746838427c3e791a4735a P 35b7b94f6a984ce88fdf4c540fec5817: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:28.285754 23054 ts_tablet_manager.cc:1918] T 793c355dcbb746838427c3e791a4735a P b07ee522f96d4f4db5788b2bce6d8afc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 16:58:28.285808 23054 log.cc:1196] T 793c355dcbb746838427c3e791a4735a P b07ee522f96d4f4db5788b2bce6d8afc: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-0/wal/wals/793c355dcbb746838427c3e791a4735a
I20260619 16:58:28.285985 23054 ts_tablet_manager.cc:1939] T 793c355dcbb746838427c3e791a4735a P b07ee522f96d4f4db5788b2bce6d8afc: Deleting consensus metadata
I20260619 16:58:28.286391 23053 ts_tablet_manager.cc:1918] T 793c355dcbb746838427c3e791a4735a P 35b7b94f6a984ce88fdf4c540fec5817: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 16:58:28.286442 23053 log.cc:1196] T 793c355dcbb746838427c3e791a4735a P 35b7b94f6a984ce88fdf4c540fec5817: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-1/wal/wals/793c355dcbb746838427c3e791a4735a
I20260619 16:58:28.286554 22507 catalog_manager.cc:4841] TS b07ee522f96d4f4db5788b2bce6d8afc (127.20.231.129:46431): tablet 793c355dcbb746838427c3e791a4735a (table test-table [id=668338ad65f24d66881441c3e921347c]) successfully deleted
I20260619 16:58:28.286680 23053 ts_tablet_manager.cc:1939] T 793c355dcbb746838427c3e791a4735a P 35b7b94f6a984ce88fdf4c540fec5817: Deleting consensus metadata
I20260619 16:58:28.286988 23052 ts_tablet_manager.cc:1918] T 793c355dcbb746838427c3e791a4735a P 89b668bda708492c8caf7dcbaad7f693: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 16:58:28.287046 23052 log.cc:1196] T 793c355dcbb746838427c3e791a4735a P 89b668bda708492c8caf7dcbaad7f693: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781888301345144-21406-0/minicluster-data/ts-2/wal/wals/793c355dcbb746838427c3e791a4735a
I20260619 16:58:28.287112 22509 catalog_manager.cc:4841] TS 35b7b94f6a984ce88fdf4c540fec5817 (127.20.231.130:33811): tablet 793c355dcbb746838427c3e791a4735a (table test-table [id=668338ad65f24d66881441c3e921347c]) successfully deleted
I20260619 16:58:28.287281 23052 ts_tablet_manager.cc:1939] T 793c355dcbb746838427c3e791a4735a P 89b668bda708492c8caf7dcbaad7f693: Deleting consensus metadata
I20260619 16:58:28.287704 22509 catalog_manager.cc:4841] TS 89b668bda708492c8caf7dcbaad7f693 (127.20.231.131:40031): tablet 793c355dcbb746838427c3e791a4735a (table test-table [id=668338ad65f24d66881441c3e921347c]) successfully deleted
Jun 19 16:58:28 dist-test-slave-vnbj krb5kdc[22477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888307, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.231.129@KRBTEST.COM
I20260619 16:58:28.298041 23025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:28.287925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36216 (local address 127.20.231.129:46431)
0619 16:58:28.287980 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:28.287982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:28.288049 (+ 67us) server_negotiation.cc:407] Connection header received
0619 16:58:28.288140 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:28.288141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:28.288156 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:28.288227 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:28.289045 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:28.289672 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:28.290510 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:28.290736 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:28.290895 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:28.294376 (+ 3481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:28.294390 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:28.294391 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:28.294401 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:28.296666 (+ 2265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:28.297093 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:28.297098 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:28.297099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:28.297176 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:28.297420 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:28.297421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:28.297422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:28.297626 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:28.297792 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:28.297841 (+ 49us) server_negotiation.cc:299] Negotiation successful
0619 16:58:28.297932 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
W20260619 16:58:28.298312 22719 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36216
I20260619 16:58:28.307053 22583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:28.299428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54766 (local address 127.20.231.190:33825)
0619 16:58:28.299471 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:28.299473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:28.299501 (+ 28us) server_negotiation.cc:407] Connection header received
0619 16:58:28.299585 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:28.299585 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:28.299595 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:28.299678 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:28.300463 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:28.301011 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:28.302210 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:28.302401 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:28.302459 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:28.304262 (+ 1803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:28.304263 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:28.304264 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:28.304271 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:28.305741 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:28.306157 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:28.306158 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:28.306159 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:28.306220 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:28.306469 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:28.306470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:28.306470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:28.306590 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:28.306675 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:28.306827 (+ 152us) server_negotiation.cc:299] Negotiation successful
0619 16:58:28.306962 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260619 16:58:28.307281 22521 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54766
I20260619 16:58:28.307955 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 22594
I20260619 16:58:28.316094 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 22734
I20260619 16:58:28.321465 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 22874
I20260619 16:58:28.326982 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 22490
2026-06-19T16:58:28Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2622 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:58:28 dist-test-slave-vnbj krb5kdc[23092](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:58:28 dist-test-slave-vnbj krb5kdc[23092](info): set up 2 sockets
Jun 19 16:58:28 dist-test-slave-vnbj krb5kdc[23092](info): commencing operation
krb5kdc: starting...
W20260619 16:58:29.398646 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 16:58:29 dist-test-slave-vnbj krb5kdc[23092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:58:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:58:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.231.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
I20260619 16:58:29.518532 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:45859
--webserver_interface=127.20.231.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:38803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:45859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:29.601842 23105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:29.602054 23105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:29.602077 23105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:29.603272 23105 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:29.603314 23105 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 16:58:29.603327 23105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:29.603340 23105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:29.603353 23105 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:29.603365 23105 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:58:29.604773 23105 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:38803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:45859
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:45859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:29.604964 23105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:29.605130 23105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:29.610229 23113 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:29.610324 23111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:29.610262 23110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:29.610296 23105 server_base.cc:1034] running on GCE node
I20260619 16:58:29.610591 23105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:29.610983 23105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:29.612170 23105 hybrid_clock.cc:648] HybridClock initialized: now 1781888309612149 us; error 33 us; skew 500 ppm
Jun 19 16:58:29 dist-test-slave-vnbj krb5kdc[23092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888309, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:29.614956 23105 init.cc:377] Logged in from keytab as kudu/127.20.231.190@KRBTEST.COM (short username kudu)
I20260619 16:58:29.615446 23105 webserver.cc:460] Webserver started at http://127.20.231.190:36355/ using document root <none> and password file <none>
I20260619 16:58:29.615676 23105 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:29.615753 23105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:29.615912 23105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:29.616999 23105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "d95249b6e2324671b047117583f5f601"
format_stamp: "Formatted at 2026-06-19 16:58:29 on dist-test-slave-vnbj"
server_key: "08c9d907af8632842b964b3d8f9b6102"
server_key_iv: "06a9136418330f42e4f0f3b4f305f152"
server_key_version: "encryptionkey@0"
I20260619 16:58:29.617374 23105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "d95249b6e2324671b047117583f5f601"
format_stamp: "Formatted at 2026-06-19 16:58:29 on dist-test-slave-vnbj"
server_key: "08c9d907af8632842b964b3d8f9b6102"
server_key_iv: "06a9136418330f42e4f0f3b4f305f152"
server_key_version: "encryptionkey@0"
I20260619 16:58:29.618712 23105 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:58:29.619652 23120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:29.619843 23105 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:58:29.619943 23105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "d95249b6e2324671b047117583f5f601"
format_stamp: "Formatted at 2026-06-19 16:58:29 on dist-test-slave-vnbj"
server_key: "08c9d907af8632842b964b3d8f9b6102"
server_key_iv: "06a9136418330f42e4f0f3b4f305f152"
server_key_version: "encryptionkey@0"
I20260619 16:58:29.620028 23105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:29.633250 23105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:29.636477 23105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:29.636659 23105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:29.641439 23105 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:45859
I20260619 16:58:29.641556 23182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:45859 every 8 connection(s)
I20260619 16:58:29.641875 23105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 16:58:29.642549 23183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:29.643596 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 23105
I20260619 16:58:29.643803 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 16:58:29.643985 21406 external_mini_cluster.cc:1428] Setting key 22e3f32d85ac18ae01bc6117a5b14b28
I20260619 16:58:29.645447 23183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d95249b6e2324671b047117583f5f601: Bootstrap starting.
I20260619 16:58:29.646093 23183 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d95249b6e2324671b047117583f5f601: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:29.646423 23183 log.cc:824] T 00000000000000000000000000000000 P d95249b6e2324671b047117583f5f601: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:29.647212 23183 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d95249b6e2324671b047117583f5f601: No bootstrap required, opened a new log
I20260619 16:58:29.649502 23183 raft_consensus.cc:348] T 00000000000000000000000000000000 P d95249b6e2324671b047117583f5f601 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d95249b6e2324671b047117583f5f601" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 45859 } }
I20260619 16:58:29.649698 23183 raft_consensus.cc:374] T 00000000000000000000000000000000 P d95249b6e2324671b047117583f5f601 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:29.649726 23183 raft_consensus.cc:729] T 00000000000000000000000000000000 P d95249b6e2324671b047117583f5f601 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d95249b6e2324671b047117583f5f601, State: Initialized, Role: FOLLOWER
I20260619 16:58:29.649855 23183 consensus_queue.cc:260] T 00000000000000000000000000000000 P d95249b6e2324671b047117583f5f601 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d95249b6e2324671b047117583f5f601" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 45859 } }
I20260619 16:58:29.649927 23183 raft_consensus.cc:388] T 00000000000000000000000000000000 P d95249b6e2324671b047117583f5f601 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:58:29.649977 23183 raft_consensus.cc:482] T 00000000000000000000000000000000 P d95249b6e2324671b047117583f5f601 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:58:29.650027 23183 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d95249b6e2324671b047117583f5f601 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:29.650591 23183 raft_consensus.cc:504] T 00000000000000000000000000000000 P d95249b6e2324671b047117583f5f601 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d95249b6e2324671b047117583f5f601" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 45859 } }
I20260619 16:58:29.650702 23183 leader_election.cc:302] T 00000000000000000000000000000000 P d95249b6e2324671b047117583f5f601 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d95249b6e2324671b047117583f5f601; no voters:
I20260619 16:58:29.650913 23183 leader_election.cc:288] T 00000000000000000000000000000000 P d95249b6e2324671b047117583f5f601 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:58:29.650970 23188 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d95249b6e2324671b047117583f5f601 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:29.651115 23188 raft_consensus.cc:686] T 00000000000000000000000000000000 P d95249b6e2324671b047117583f5f601 [term 1 LEADER]: Becoming Leader. State: Replica: d95249b6e2324671b047117583f5f601, State: Running, Role: LEADER
I20260619 16:58:29.651211 23188 consensus_queue.cc:237] T 00000000000000000000000000000000 P d95249b6e2324671b047117583f5f601 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d95249b6e2324671b047117583f5f601" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 45859 } }
I20260619 16:58:29.651216 23183 sys_catalog.cc:564] T 00000000000000000000000000000000 P d95249b6e2324671b047117583f5f601 [sys.catalog]: configured and running, proceeding with master startup.
Jun 19 16:58:29 dist-test-slave-vnbj krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:29.651546 23188 sys_catalog.cc:455] T 00000000000000000000000000000000 P d95249b6e2324671b047117583f5f601 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d95249b6e2324671b047117583f5f601. Latest consensus state: current_term: 1 leader_uuid: "d95249b6e2324671b047117583f5f601" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d95249b6e2324671b047117583f5f601" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 45859 } } }
I20260619 16:58:29.651638 23188 sys_catalog.cc:458] T 00000000000000000000000000000000 P d95249b6e2324671b047117583f5f601 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:29.651561 23189 sys_catalog.cc:455] T 00000000000000000000000000000000 P d95249b6e2324671b047117583f5f601 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d95249b6e2324671b047117583f5f601" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d95249b6e2324671b047117583f5f601" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 45859 } } }
I20260619 16:58:29.651818 23189 sys_catalog.cc:458] T 00000000000000000000000000000000 P d95249b6e2324671b047117583f5f601 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:29.651939 23193 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:58:29.652453 23193 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:58:29.655212 23193 catalog_manager.cc:1347] Generated new cluster ID: aab86a989d0d4ee0aca0fc7b5c95e081
I20260619 16:58:29.655272 23193 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:58:29.656005 23186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:29.644532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35370 (local address 127.20.231.190:45859)
0619 16:58:29.644715 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:29.644725 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:29.644761 (+ 36us) server_negotiation.cc:407] Connection header received
0619 16:58:29.645287 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:29.645296 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:29.645365 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:29.645532 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:29.646145 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:29.647552 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:29.648295 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:29.648936 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:29.649056 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:29.651937 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:29.651942 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:29.651943 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:29.651955 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:29.653885 (+ 1930us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:29.654330 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:29.654334 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:29.654334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:29.654495 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:29.654751 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:29.654753 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:29.654754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:29.655104 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:29.655239 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:29.655533 (+ 294us) server_negotiation.cc:299] Negotiation successful
0619 16:58:29.655675 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":78,"threads_started":1}
I20260619 16:58:29.683332 23193 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:58:29.683880 23193 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:58:29.686702 23193 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d95249b6e2324671b047117583f5f601: Generated new TSK 0
I20260619 16:58:29.686923 23193 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.231.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab.
Entry for principal kudu/127.20.231.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab.
I20260619 16:58:29.721208 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.129:0
--local_ip_for_outbound_sockets=127.20.231.129
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:45859
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:38803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:29.826452 23209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:29.826668 23209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:29.826694 23209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:29.827937 23209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:29.828004 23209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:29.828027 23209 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 16:58:29.828039 23209 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 16:58:29.828081 23209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.129
I20260619 16:58:29.829591 23209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:38803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.231.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.231.190:45859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.129
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:29.829854 23209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:29.830092 23209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 16:58:29.835471 23209 server_base.cc:1034] running on GCE node
W20260619 16:58:29.835443 23217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:29.835451 23214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:29.835451 23215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:29.836006 23209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:29.836272 23209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:29.837422 23209 hybrid_clock.cc:648] HybridClock initialized: now 1781888309837408 us; error 28 us; skew 500 ppm
Jun 19 16:58:29 dist-test-slave-vnbj krb5kdc[23092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888309, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:29.840096 23209 init.cc:377] Logged in from keytab as kudu/127.20.231.129@KRBTEST.COM (short username kudu)
I20260619 16:58:29.840591 23209 webserver.cc:460] Webserver started at http://127.20.231.129:41967/ using document root <none> and password file <none>
I20260619 16:58:29.840812 23209 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:29.840885 23209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:29.841032 23209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:29.842104 23209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-0/data/instance:
uuid: "c84391626820478bbb21215c50fb899a"
format_stamp: "Formatted at 2026-06-19 16:58:29 on dist-test-slave-vnbj"
server_key: "263ada911b40e69417b1f513a64671f3"
server_key_iv: "018e0a79e12c38da24114f06dd7f4653"
server_key_version: "encryptionkey@0"
I20260619 16:58:29.842495 23209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance:
uuid: "c84391626820478bbb21215c50fb899a"
format_stamp: "Formatted at 2026-06-19 16:58:29 on dist-test-slave-vnbj"
server_key: "263ada911b40e69417b1f513a64671f3"
server_key_iv: "018e0a79e12c38da24114f06dd7f4653"
server_key_version: "encryptionkey@0"
I20260619 16:58:29.843910 23209 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:58:29.844674 23224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:29.844911 23209 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:58:29.845010 23209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-0/wal
uuid: "c84391626820478bbb21215c50fb899a"
format_stamp: "Formatted at 2026-06-19 16:58:29 on dist-test-slave-vnbj"
server_key: "263ada911b40e69417b1f513a64671f3"
server_key_iv: "018e0a79e12c38da24114f06dd7f4653"
server_key_version: "encryptionkey@0"
I20260619 16:58:29.845086 23209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:29.862687 23209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:29.864913 23209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:29.865082 23209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:29.865674 23209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:29.865738 23209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:29.865780 23209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:29.865808 23209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 16:58:29 dist-test-slave-vnbj krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888309, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:29.873572 23209 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.129:35139
I20260619 16:58:29.873647 23339 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.129:35139 every 8 connection(s)
I20260619 16:58:29.874010 23209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
I20260619 16:58:29.876854 23186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:29.866238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:47671 (local address 127.20.231.190:45859)
0619 16:58:29.866305 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:29.866307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:29.866523 (+ 216us) server_negotiation.cc:407] Connection header received
0619 16:58:29.867352 (+ 829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:29.867353 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:29.867369 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:29.867445 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:29.868482 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:29.869266 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:29.870712 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:29.871109 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:29.871197 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:29.873459 (+ 2262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:29.873461 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:29.873462 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:29.873471 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:29.875171 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:29.875695 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:29.875696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:29.875697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:29.875759 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:29.876092 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:29.876094 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:29.876094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:29.876252 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:29.876339 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:29.876666 (+ 327us) server_negotiation.cc:299] Negotiation successful
0619 16:58:29.876784 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 16:58:29.876885 23233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:29.866220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:45859 (local address 127.20.231.129:47671)
0619 16:58:29.866384 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:29.866420 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:29.867201 (+ 781us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:29.867686 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:29.867690 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:29.868066 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:29.868344 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:29.868352 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:29.869402 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:29.869403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:29.870608 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:29.870611 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:29.871221 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:29.871222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:29.871349 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:29.871913 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:29.871922 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:29.873345 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:29.875287 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:29.875293 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:29.875295 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:29.875517 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:29.875864 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:29.875865 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:29.875865 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:29.875979 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:29.876329 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:29.876334 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:29.876451 (+ 117us) client_negotiation.cc:769] Sending connection context
0619 16:58:29.876537 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 16:58:29.876722 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":95,"threads_started":1}
I20260619 16:58:29.877215 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 23209
I20260619 16:58:29.877292 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance
I20260619 16:58:29.877437 21406 external_mini_cluster.cc:1428] Setting key 0c10f0bb316accbe3d9bdf398c6c5bd9
I20260619 16:58:29.878134 23340 heartbeater.cc:347] Connected to a master server at 127.20.231.190:45859
I20260619 16:58:29.878290 23340 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:29.878513 23340 heartbeater.cc:511] Master 127.20.231.190:45859 requested a full tablet report, sending...
I20260619 16:58:29.879136 23137 ts_manager.cc:194] Registered new tserver with Master: c84391626820478bbb21215c50fb899a (127.20.231.129:35139)
I20260619 16:58:29.880051 23137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.129@KRBTEST.COM'} at 127.20.231.129:47671
WARNING: no policy specified for kudu/127.20.231.130@KRBTEST.COM; defaulting to no policy
I20260619 16:58:29.889066 23186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:29.879373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:38473 (local address 127.20.231.190:45859)
0619 16:58:29.879419 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:29.879422 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:29.879540 (+ 118us) server_negotiation.cc:407] Connection header received
0619 16:58:29.879689 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:29.879690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:29.879704 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:29.879745 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:29.883015 (+ 3270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:29.883503 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:29.884780 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:29.884964 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:29.885023 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:29.886285 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:29.886286 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:29.886286 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:29.886292 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:29.887895 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:29.888261 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:29.888262 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:29.888262 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:29.888311 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:29.888581 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:29.888582 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:29.888582 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:29.888678 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:29.888728 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:29.888940 (+ 212us) server_negotiation.cc:299] Negotiation successful
0619 16:58:29.889004 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 16:58:29.889071 23346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:29.879317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:45859 (local address 127.20.231.129:38473)
0619 16:58:29.879468 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:29.879481 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:29.879574 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:29.879821 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:29.879822 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:29.880130 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:29.880525 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:29.880528 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:29.883634 (+ 3106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:29.883636 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:29.884679 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:29.884683 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:29.885052 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:29.885054 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:29.885155 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:29.885680 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:29.885687 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:29.886191 (+ 504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:29.887983 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:29.887983 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:29.887984 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:29.888187 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:29.888397 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:29.888397 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:29.888398 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:29.888512 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:29.888782 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:29.888783 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:29.888826 (+ 43us) client_negotiation.cc:769] Sending connection context
0619 16:58:29.888912 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 16:58:29.889000 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":75,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab.
Entry for principal kudu/127.20.231.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab.
I20260619 16:58:29.906033 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.130:0
--local_ip_for_outbound_sockets=127.20.231.130
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:45859
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:38803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:29.989089 23349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:29.989302 23349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:29.989328 23349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:29.990545 23349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:29.990592 23349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:29.990619 23349 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 16:58:29.990631 23349 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 16:58:29.990696 23349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.130
I20260619 16:58:29.992259 23349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:38803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.231.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.231.190:45859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.130
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:29.992545 23349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:29.992889 23349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:29.998868 23355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:29.998862 23357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:29.999017 23354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:29.999138 23349 server_base.cc:1034] running on GCE node
I20260619 16:58:29.999370 23349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:29.999722 23349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:30.000916 23349 hybrid_clock.cc:648] HybridClock initialized: now 1781888310000893 us; error 38 us; skew 500 ppm
Jun 19 16:58:30 dist-test-slave-vnbj krb5kdc[23092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888310, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:30.003764 23349 init.cc:377] Logged in from keytab as kudu/127.20.231.130@KRBTEST.COM (short username kudu)
I20260619 16:58:30.004256 23349 webserver.cc:460] Webserver started at http://127.20.231.130:38595/ using document root <none> and password file <none>
I20260619 16:58:30.004518 23349 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:30.004590 23349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:30.004739 23349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:30.005867 23349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-1/data/instance:
uuid: "af047fc88d734af0b2b6aad65d01bdb0"
format_stamp: "Formatted at 2026-06-19 16:58:30 on dist-test-slave-vnbj"
server_key: "ee4157f17321ccfd7f56ee7d3b7570a1"
server_key_iv: "781977af0e0df4559c667377f7157c2d"
server_key_version: "encryptionkey@0"
I20260619 16:58:30.006273 23349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance:
uuid: "af047fc88d734af0b2b6aad65d01bdb0"
format_stamp: "Formatted at 2026-06-19 16:58:30 on dist-test-slave-vnbj"
server_key: "ee4157f17321ccfd7f56ee7d3b7570a1"
server_key_iv: "781977af0e0df4559c667377f7157c2d"
server_key_version: "encryptionkey@0"
I20260619 16:58:30.007725 23349 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:58:30.008461 23364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:30.008625 23349 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:58:30.008728 23349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-1/wal
uuid: "af047fc88d734af0b2b6aad65d01bdb0"
format_stamp: "Formatted at 2026-06-19 16:58:30 on dist-test-slave-vnbj"
server_key: "ee4157f17321ccfd7f56ee7d3b7570a1"
server_key_iv: "781977af0e0df4559c667377f7157c2d"
server_key_version: "encryptionkey@0"
I20260619 16:58:30.008811 23349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:30.020664 23349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:30.023236 23349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:30.023429 23349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:30.024021 23349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:30.024089 23349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:30.024132 23349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:30.024154 23349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:30.031013 23349 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.130:34463
I20260619 16:58:30.031111 23479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.130:34463 every 8 connection(s)
I20260619 16:58:30.031425 23349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
Jun 19 16:58:30 dist-test-slave-vnbj krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888310, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:30.035144 23186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.024403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:33163 (local address 127.20.231.190:45859)
0619 16:58:30.024469 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:30.024472 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:30.024714 (+ 242us) server_negotiation.cc:407] Connection header received
0619 16:58:30.025262 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:30.025263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:30.025279 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:30.025378 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:30.026321 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.027227 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.028961 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.029265 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.029400 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.032227 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:30.032228 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:30.032229 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:30.032236 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:30.033576 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:30.034057 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:30.034058 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:30.034059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:30.034123 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:30.034389 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:30.034391 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:30.034391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:30.034535 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:30.034612 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:30.034956 (+ 344us) server_negotiation.cc:299] Negotiation successful
0619 16:58:30.035043 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 16:58:30.035382 23373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.024401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:45859 (local address 127.20.231.130:33163)
0619 16:58:30.024580 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:30.024621 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:30.025161 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:30.025475 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:30.025481 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:30.025902 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:30.026202 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.026209 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.027335 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.027336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.028865 (+ 1529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.028868 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.029411 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.029412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.029573 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.030236 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:30.030255 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:30.032108 (+ 1853us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:30.033672 (+ 1564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:30.033680 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:30.033683 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:30.033970 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:30.034193 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:30.034194 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:30.034194 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:30.034319 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:30.034612 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:30.034620 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:30.034768 (+ 148us) client_negotiation.cc:769] Sending connection context
0619 16:58:30.034956 (+ 188us) client_negotiation.cc:240] Negotiation successful
0619 16:58:30.035131 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":113,"threads_started":1}
I20260619 16:58:30.035907 23480 heartbeater.cc:347] Connected to a master server at 127.20.231.190:45859
I20260619 16:58:30.036017 23480 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:30.036206 23480 heartbeater.cc:511] Master 127.20.231.190:45859 requested a full tablet report, sending...
I20260619 16:58:30.036661 23135 ts_manager.cc:194] Registered new tserver with Master: af047fc88d734af0b2b6aad65d01bdb0 (127.20.231.130:34463)
I20260619 16:58:30.037245 23135 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.130@KRBTEST.COM'} at 127.20.231.130:33163
I20260619 16:58:30.041190 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 23349
I20260619 16:58:30.041303 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance
I20260619 16:58:30.041507 21406 external_mini_cluster.cc:1428] Setting key c46b7ddb590be6d7557cc457115f5a8b
I20260619 16:58:30.043058 23186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.037081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:40407 (local address 127.20.231.190:45859)
0619 16:58:30.037129 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:30.037131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:30.037212 (+ 81us) server_negotiation.cc:407] Connection header received
0619 16:58:30.037266 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:30.037267 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:30.037278 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:30.037355 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:30.037736 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.038370 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.039169 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.039360 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.039416 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.040307 (+ 891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:30.040308 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:30.040308 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:30.040314 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:30.041836 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:30.042192 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:30.042193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:30.042193 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:30.042242 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:30.042523 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:30.042524 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:30.042524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:30.042670 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:30.042727 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:30.042935 (+ 208us) server_negotiation.cc:299] Negotiation successful
0619 16:58:30.042996 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 16:58:30.043205 23486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.036992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:45859 (local address 127.20.231.130:40407)
0619 16:58:30.037136 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:30.037147 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:30.037208 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:30.037391 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:30.037392 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:30.037579 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:30.037663 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.037665 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.038460 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.038461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.039100 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.039102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.039426 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.039427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.039481 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.039948 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:30.039953 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:30.040231 (+ 278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:30.041894 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:30.041895 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:30.041896 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:30.042112 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:30.042344 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:30.042344 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:30.042345 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:30.042429 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:30.042741 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:30.042742 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:30.042794 (+ 52us) client_negotiation.cc:769] Sending connection context
0619 16:58:30.042908 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 16:58:30.043038 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":3,"thread_start_us":96,"threads_started":1}
WARNING: no policy specified for kudu/127.20.231.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab.
Entry for principal kudu/127.20.231.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab.
I20260619 16:58:30.069717 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.131:0
--local_ip_for_outbound_sockets=127.20.231.131
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:45859
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:38803
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:30.152720 23489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:30.152984 23489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:30.153029 23489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:30.154219 23489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:30.154296 23489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:30.154335 23489 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 16:58:30.154382 23489 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 16:58:30.154422 23489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.131
I20260619 16:58:30.155982 23489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:38803
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.231.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.231.190:45859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.131
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:30.156250 23489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:30.156484 23489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:30.161844 23494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:30.161916 23495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:30.161896 23497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:30.162197 23489 server_base.cc:1034] running on GCE node
I20260619 16:58:30.162413 23489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:30.162730 23489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:30.164041 23489 hybrid_clock.cc:648] HybridClock initialized: now 1781888310164049 us; error 53 us; skew 500 ppm
Jun 19 16:58:30 dist-test-slave-vnbj krb5kdc[23092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888310, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:30.166783 23489 init.cc:377] Logged in from keytab as kudu/127.20.231.131@KRBTEST.COM (short username kudu)
I20260619 16:58:30.167239 23489 webserver.cc:460] Webserver started at http://127.20.231.131:41659/ using document root <none> and password file <none>
I20260619 16:58:30.167466 23489 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:30.167534 23489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:30.167681 23489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:30.168795 23489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-2/data/instance:
uuid: "b17be09a87944e6f829f6198f4f21b8c"
format_stamp: "Formatted at 2026-06-19 16:58:30 on dist-test-slave-vnbj"
server_key: "401799a1a78232ce50d6cb366a22fb76"
server_key_iv: "029b939443afa6f4a11e2b609087bbd8"
server_key_version: "encryptionkey@0"
I20260619 16:58:30.169163 23489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance:
uuid: "b17be09a87944e6f829f6198f4f21b8c"
format_stamp: "Formatted at 2026-06-19 16:58:30 on dist-test-slave-vnbj"
server_key: "401799a1a78232ce50d6cb366a22fb76"
server_key_iv: "029b939443afa6f4a11e2b609087bbd8"
server_key_version: "encryptionkey@0"
I20260619 16:58:30.170507 23489 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:58:30.171376 23504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:30.171546 23489 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:30.171648 23489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-2/wal
uuid: "b17be09a87944e6f829f6198f4f21b8c"
format_stamp: "Formatted at 2026-06-19 16:58:30 on dist-test-slave-vnbj"
server_key: "401799a1a78232ce50d6cb366a22fb76"
server_key_iv: "029b939443afa6f4a11e2b609087bbd8"
server_key_version: "encryptionkey@0"
I20260619 16:58:30.171721 23489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:30.194115 23489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:30.196326 23489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:30.196496 23489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:30.197103 23489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:30.197170 23489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:30.197216 23489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:30.197242 23489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:30.204484 23489 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.131:34207
I20260619 16:58:30.204555 23619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.131:34207 every 8 connection(s)
I20260619 16:58:30.204865 23489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
Jun 19 16:58:30 dist-test-slave-vnbj krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888310, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:30.208495 23186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.197756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:48451 (local address 127.20.231.190:45859)
0619 16:58:30.197830 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:30.197833 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:30.198106 (+ 273us) server_negotiation.cc:407] Connection header received
0619 16:58:30.198680 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:30.198682 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:30.198694 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:30.198737 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:30.199524 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.200037 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.201521 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.201849 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.201927 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.205466 (+ 3539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:30.205468 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:30.205468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:30.205477 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:30.207060 (+ 1583us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:30.207506 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:30.207508 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:30.207508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:30.207568 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:30.207836 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:30.207837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:30.207837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:30.207960 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:30.208032 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:30.208324 (+ 292us) server_negotiation.cc:299] Negotiation successful
0619 16:58:30.208405 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 16:58:30.208712 23513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.197737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:45859 (local address 127.20.231.131:48451)
0619 16:58:30.197972 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:30.198016 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:30.198564 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:30.198924 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:30.198928 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:30.199186 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:30.199420 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.199426 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.200134 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.200135 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.201448 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.201451 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.201958 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.201960 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.202141 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.203413 (+ 1272us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:30.203425 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:30.205357 (+ 1932us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:30.207160 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:30.207172 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:30.207174 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:30.207403 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:30.207637 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:30.207638 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:30.207638 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:30.207759 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:30.208036 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:30.208042 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:30.208184 (+ 142us) client_negotiation.cc:769] Sending connection context
0619 16:58:30.208292 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 16:58:30.208457 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":127,"threads_started":1}
I20260619 16:58:30.209180 23620 heartbeater.cc:347] Connected to a master server at 127.20.231.190:45859
I20260619 16:58:30.209270 23620 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:30.209477 23620 heartbeater.cc:511] Master 127.20.231.190:45859 requested a full tablet report, sending...
I20260619 16:58:30.209853 23137 ts_manager.cc:194] Registered new tserver with Master: b17be09a87944e6f829f6198f4f21b8c (127.20.231.131:34207)
I20260619 16:58:30.210388 23137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.131@KRBTEST.COM'} at 127.20.231.131:48451
I20260619 16:58:30.215046 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 23489
I20260619 16:58:30.215142 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance
I20260619 16:58:30.215303 21406 external_mini_cluster.cc:1428] Setting key 6a3db38b8da818e47afce11c4008d15c
I20260619 16:58:30.215895 23626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.210337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:45859 (local address 127.20.231.131:43335)
0619 16:58:30.210470 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:30.210481 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:30.210547 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:30.210791 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:30.210791 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:30.210997 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:30.211046 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.211048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.211726 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.211727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.212294 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.212296 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.212671 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.212671 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.212751 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.213179 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:30.213184 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:30.213479 (+ 295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:30.214869 (+ 1390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:30.214870 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:30.214870 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:30.215042 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:30.215244 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:30.215244 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:30.215245 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:30.215298 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:30.215549 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:30.215550 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:30.215596 (+ 46us) client_negotiation.cc:769] Sending connection context
0619 16:58:30.215666 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 16:58:30.215779 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260619 16:58:30.215967 23186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.210436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:43335 (local address 127.20.231.190:45859)
0619 16:58:30.210489 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:30.210491 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:30.210683 (+ 192us) server_negotiation.cc:407] Connection header received
0619 16:58:30.210701 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:30.210702 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:30.210710 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:30.210786 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:30.211124 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.211609 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.212384 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.212585 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.212669 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.213569 (+ 900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:30.213571 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:30.213571 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:30.213579 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:30.214711 (+ 1132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:30.215127 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:30.215129 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:30.215129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:30.215185 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:30.215370 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:30.215371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:30.215371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:30.215482 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:30.215560 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:30.215749 (+ 189us) server_negotiation.cc:299] Negotiation successful
0619 16:58:30.215843 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 16:58:30.216398 21406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 16:58:30 dist-test-slave-vnbj krb5kdc[23092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 16:58:30 dist-test-slave-vnbj krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:30.239562 23186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.232089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35380 (local address 127.20.231.190:45859)
0619 16:58:30.232154 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:30.232156 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:30.232167 (+ 11us) server_negotiation.cc:407] Connection header received
0619 16:58:30.232218 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:30.232219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:30.232228 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:30.232300 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:30.232896 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.233397 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.234149 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.234353 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.234468 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.236836 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:30.236838 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:30.236838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:30.236846 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:30.238542 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:30.238929 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:30.238930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:30.238931 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:30.238985 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:30.239160 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:30.239160 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:30.239161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:30.239273 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:30.239349 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:30.239437 (+ 88us) server_negotiation.cc:299] Negotiation successful
0619 16:58:30.239497 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 16:58:30.241405 23137 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35380:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:58:30.243012 23137 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:58:30.250030 23641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.245732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37310 (local address 127.20.231.131:34207)
0619 16:58:30.246008 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:30.246012 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:30.246021 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:58:30.246061 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:30.246062 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:30.246075 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:30.246161 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:30.246347 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.247311 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.248800 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.249570 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.249656 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.249716 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:30.249890 (+ 174us) server_negotiation.cc:299] Negotiation successful
0619 16:58:30.249932 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":98,"threads_started":1}
I20260619 16:58:30.250021 23638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.245483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:34207 (local address 127.0.0.1:37310)
0619 16:58:30.245876 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:30.245886 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:30.245926 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:30.246170 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:30.246173 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:30.246175 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:30.246253 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.246254 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.247393 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.247394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.248704 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.248707 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.249661 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.249662 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.249756 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.249770 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 16:58:30.249874 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 16:58:30.249929 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":55,"threads_started":1}
I20260619 16:58:30.250240 23642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.245935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56216 (local address 127.20.231.130:34463)
0619 16:58:30.246085 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:30.246089 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:30.246102 (+ 13us) server_negotiation.cc:407] Connection header received
0619 16:58:30.246148 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:30.246150 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:30.246166 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:30.246256 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:30.246447 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.247675 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.248954 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.249692 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.249780 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.249832 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:30.250087 (+ 255us) server_negotiation.cc:299] Negotiation successful
0619 16:58:30.250138 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":86,"threads_started":1}
I20260619 16:58:30.250475 23640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.245647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47066 (local address 127.20.231.129:35139)
0619 16:58:30.245927 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:30.245932 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:30.246245 (+ 313us) server_negotiation.cc:407] Connection header received
0619 16:58:30.246510 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:30.246513 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:30.246529 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:30.246609 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:30.246723 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.247812 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.249143 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.249884 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.249948 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.249995 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:30.250315 (+ 320us) server_negotiation.cc:299] Negotiation successful
0619 16:58:30.250363 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":182,"threads_started":1}
I20260619 16:58:30.250239 23639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.245483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:35139 (local address 127.0.0.1:47066)
0619 16:58:30.246118 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:30.246128 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:30.246327 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:30.246619 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:30.246620 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:30.246622 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:30.246658 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.246659 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.247901 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.247902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.249058 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.249061 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.250026 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.250026 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.250079 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.250082 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:58:30.250128 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 16:58:30.250159 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":615,"thread_start_us":93,"threads_started":1}
I20260619 16:58:30.250428 23637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.245481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:34463 (local address 127.0.0.1:56216)
0619 16:58:30.245646 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:30.245685 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:30.245749 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:30.246251 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:30.246252 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:30.246254 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:30.246376 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.246378 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.247767 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.247768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.248871 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.248873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.249869 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.249870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.249947 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.249950 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:58:30.249998 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 16:58:30.250043 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":86,"threads_started":1}
I20260619 16:58:30.251276 23554 tablet_service.cc:1467] Processing CreateTablet for tablet a646a13b898747b18afac954bfe0ce44 (DEFAULT_TABLE table=test-table [id=570b34468a514f7f9f50bd96477061c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:30.251291 23272 tablet_service.cc:1467] Processing CreateTablet for tablet a646a13b898747b18afac954bfe0ce44 (DEFAULT_TABLE table=test-table [id=570b34468a514f7f9f50bd96477061c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:30.251528 23414 tablet_service.cc:1467] Processing CreateTablet for tablet a646a13b898747b18afac954bfe0ce44 (DEFAULT_TABLE table=test-table [id=570b34468a514f7f9f50bd96477061c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:30.251658 23554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a646a13b898747b18afac954bfe0ce44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:30.251658 23272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a646a13b898747b18afac954bfe0ce44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:30.251904 23414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a646a13b898747b18afac954bfe0ce44. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:30.254768 23643 tablet_bootstrap.cc:492] T a646a13b898747b18afac954bfe0ce44 P af047fc88d734af0b2b6aad65d01bdb0: Bootstrap starting.
I20260619 16:58:30.255295 23645 tablet_bootstrap.cc:492] T a646a13b898747b18afac954bfe0ce44 P c84391626820478bbb21215c50fb899a: Bootstrap starting.
I20260619 16:58:30.255541 23643 tablet_bootstrap.cc:654] T a646a13b898747b18afac954bfe0ce44 P af047fc88d734af0b2b6aad65d01bdb0: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:30.255570 23644 tablet_bootstrap.cc:492] T a646a13b898747b18afac954bfe0ce44 P b17be09a87944e6f829f6198f4f21b8c: Bootstrap starting.
I20260619 16:58:30.255861 23643 log.cc:824] T a646a13b898747b18afac954bfe0ce44 P af047fc88d734af0b2b6aad65d01bdb0: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:30.255970 23645 tablet_bootstrap.cc:654] T a646a13b898747b18afac954bfe0ce44 P c84391626820478bbb21215c50fb899a: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:30.256330 23645 log.cc:824] T a646a13b898747b18afac954bfe0ce44 P c84391626820478bbb21215c50fb899a: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:30.256446 23644 tablet_bootstrap.cc:654] T a646a13b898747b18afac954bfe0ce44 P b17be09a87944e6f829f6198f4f21b8c: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:30.256609 23643 tablet_bootstrap.cc:492] T a646a13b898747b18afac954bfe0ce44 P af047fc88d734af0b2b6aad65d01bdb0: No bootstrap required, opened a new log
I20260619 16:58:30.256688 23643 ts_tablet_manager.cc:1397] T a646a13b898747b18afac954bfe0ce44 P af047fc88d734af0b2b6aad65d01bdb0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:58:30.256807 23644 log.cc:824] T a646a13b898747b18afac954bfe0ce44 P b17be09a87944e6f829f6198f4f21b8c: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:30.257192 23645 tablet_bootstrap.cc:492] T a646a13b898747b18afac954bfe0ce44 P c84391626820478bbb21215c50fb899a: No bootstrap required, opened a new log
I20260619 16:58:30.257284 23645 ts_tablet_manager.cc:1397] T a646a13b898747b18afac954bfe0ce44 P c84391626820478bbb21215c50fb899a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 16:58:30.257616 23644 tablet_bootstrap.cc:492] T a646a13b898747b18afac954bfe0ce44 P b17be09a87944e6f829f6198f4f21b8c: No bootstrap required, opened a new log
I20260619 16:58:30.257710 23644 ts_tablet_manager.cc:1397] T a646a13b898747b18afac954bfe0ce44 P b17be09a87944e6f829f6198f4f21b8c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 16:58:30.259094 23643 raft_consensus.cc:348] T a646a13b898747b18afac954bfe0ce44 P af047fc88d734af0b2b6aad65d01bdb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:30.259294 23643 raft_consensus.cc:374] T a646a13b898747b18afac954bfe0ce44 P af047fc88d734af0b2b6aad65d01bdb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:30.259332 23643 raft_consensus.cc:729] T a646a13b898747b18afac954bfe0ce44 P af047fc88d734af0b2b6aad65d01bdb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af047fc88d734af0b2b6aad65d01bdb0, State: Initialized, Role: FOLLOWER
I20260619 16:58:30.259347 23644 raft_consensus.cc:348] T a646a13b898747b18afac954bfe0ce44 P b17be09a87944e6f829f6198f4f21b8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:30.259533 23644 raft_consensus.cc:374] T a646a13b898747b18afac954bfe0ce44 P b17be09a87944e6f829f6198f4f21b8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:30.259486 23643 consensus_queue.cc:260] T a646a13b898747b18afac954bfe0ce44 P af047fc88d734af0b2b6aad65d01bdb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:30.259577 23644 raft_consensus.cc:729] T a646a13b898747b18afac954bfe0ce44 P b17be09a87944e6f829f6198f4f21b8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b17be09a87944e6f829f6198f4f21b8c, State: Initialized, Role: FOLLOWER
I20260619 16:58:30.259482 23645 raft_consensus.cc:348] T a646a13b898747b18afac954bfe0ce44 P c84391626820478bbb21215c50fb899a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:30.259624 23645 raft_consensus.cc:374] T a646a13b898747b18afac954bfe0ce44 P c84391626820478bbb21215c50fb899a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:30.259678 23645 raft_consensus.cc:729] T a646a13b898747b18afac954bfe0ce44 P c84391626820478bbb21215c50fb899a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c84391626820478bbb21215c50fb899a, State: Initialized, Role: FOLLOWER
I20260619 16:58:30.259734 23644 consensus_queue.cc:260] T a646a13b898747b18afac954bfe0ce44 P b17be09a87944e6f829f6198f4f21b8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:30.259778 23643 ts_tablet_manager.cc:1428] T a646a13b898747b18afac954bfe0ce44 P af047fc88d734af0b2b6aad65d01bdb0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 16:58:30.259802 23645 consensus_queue.cc:260] T a646a13b898747b18afac954bfe0ce44 P c84391626820478bbb21215c50fb899a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:30.259903 23480 heartbeater.cc:503] Master 127.20.231.190:45859 was elected leader, sending a full tablet report...
I20260619 16:58:30.259982 23644 ts_tablet_manager.cc:1428] T a646a13b898747b18afac954bfe0ce44 P b17be09a87944e6f829f6198f4f21b8c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 16:58:30.260051 23645 ts_tablet_manager.cc:1428] T a646a13b898747b18afac954bfe0ce44 P c84391626820478bbb21215c50fb899a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 16:58:30.260105 23620 heartbeater.cc:503] Master 127.20.231.190:45859 was elected leader, sending a full tablet report...
I20260619 16:58:30.260164 23340 heartbeater.cc:503] Master 127.20.231.190:45859 was elected leader, sending a full tablet report...
W20260619 16:58:30.282078 23481 tablet.cc:2369] T a646a13b898747b18afac954bfe0ce44 P af047fc88d734af0b2b6aad65d01bdb0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:58:30.289176 23651 raft_consensus.cc:482] T a646a13b898747b18afac954bfe0ce44 P c84391626820478bbb21215c50fb899a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:58:30.289304 23651 raft_consensus.cc:504] T a646a13b898747b18afac954bfe0ce44 P c84391626820478bbb21215c50fb899a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:30.289726 23651 leader_election.cc:288] T a646a13b898747b18afac954bfe0ce44 P c84391626820478bbb21215c50fb899a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af047fc88d734af0b2b6aad65d01bdb0 (127.20.231.130:34463), b17be09a87944e6f829f6198f4f21b8c (127.20.231.131:34207)
I20260619 16:58:30.293728 23642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.290016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:39937 (local address 127.20.231.130:34463)
0619 16:58:30.290061 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:30.290063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:30.290106 (+ 43us) server_negotiation.cc:407] Connection header received
0619 16:58:30.290133 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:30.290134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:30.290143 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:30.290193 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:30.290383 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.291182 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.292735 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.293329 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.293408 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.293416 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:30.293611 (+ 195us) server_negotiation.cc:299] Negotiation successful
0619 16:58:30.293657 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 16:58:30.293747 23652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.289937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:34207 (local address 127.20.231.129:34747)
0619 16:58:30.290146 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:30.290154 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:30.290229 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:30.290388 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:30.290389 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:30.290391 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:30.290453 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.290454 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.291343 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.291344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.292610 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.292613 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.293428 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.293429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.293514 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.293518 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:58:30.293584 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 16:58:30.293643 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"mutex_wait_us":61,"thread_start_us":105,"threads_started":1}
I20260619 16:58:30.293807 23641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.290043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:34747 (local address 127.20.231.131:34207)
0619 16:58:30.290100 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:30.290102 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:30.290266 (+ 164us) server_negotiation.cc:407] Connection header received
0619 16:58:30.290295 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:30.290296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:30.290303 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:30.290370 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:30.290516 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.291263 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.292735 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.293321 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.293408 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.293416 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:30.293613 (+ 197us) server_negotiation.cc:299] Negotiation successful
0619 16:58:30.293733 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 16:58:30.294081 23233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.289937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:34463 (local address 127.20.231.129:39937)
0619 16:58:30.290001 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:30.290014 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:30.290057 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:30.290228 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:30.290229 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:30.290231 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:30.290309 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.290310 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.291276 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.291277 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.292610 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.292613 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.293428 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.293429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.293521 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.293523 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 16:58:30.293584 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 16:58:30.293643 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44,"mutex_wait_us":65}
I20260619 16:58:30.294247 23574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a646a13b898747b18afac954bfe0ce44" candidate_uuid: "c84391626820478bbb21215c50fb899a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17be09a87944e6f829f6198f4f21b8c" is_pre_election: true
I20260619 16:58:30.294414 23574 raft_consensus.cc:2454] T a646a13b898747b18afac954bfe0ce44 P b17be09a87944e6f829f6198f4f21b8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c84391626820478bbb21215c50fb899a in term 0.
I20260619 16:58:30.294380 23434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a646a13b898747b18afac954bfe0ce44" candidate_uuid: "c84391626820478bbb21215c50fb899a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af047fc88d734af0b2b6aad65d01bdb0" is_pre_election: true
I20260619 16:58:30.294498 23434 raft_consensus.cc:2454] T a646a13b898747b18afac954bfe0ce44 P af047fc88d734af0b2b6aad65d01bdb0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c84391626820478bbb21215c50fb899a in term 0.
I20260619 16:58:30.294786 23226 leader_election.cc:302] T a646a13b898747b18afac954bfe0ce44 P c84391626820478bbb21215c50fb899a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b17be09a87944e6f829f6198f4f21b8c, c84391626820478bbb21215c50fb899a; no voters:
I20260619 16:58:30.294934 23651 raft_consensus.cc:2781] T a646a13b898747b18afac954bfe0ce44 P c84391626820478bbb21215c50fb899a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:58:30.294982 23651 raft_consensus.cc:482] T a646a13b898747b18afac954bfe0ce44 P c84391626820478bbb21215c50fb899a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:58:30.295006 23651 raft_consensus.cc:3037] T a646a13b898747b18afac954bfe0ce44 P c84391626820478bbb21215c50fb899a [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:30.295871 23651 raft_consensus.cc:504] T a646a13b898747b18afac954bfe0ce44 P c84391626820478bbb21215c50fb899a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:30.296087 23651 leader_election.cc:288] T a646a13b898747b18afac954bfe0ce44 P c84391626820478bbb21215c50fb899a [CANDIDATE]: Term 1 election: Requested vote from peers af047fc88d734af0b2b6aad65d01bdb0 (127.20.231.130:34463), b17be09a87944e6f829f6198f4f21b8c (127.20.231.131:34207)
I20260619 16:58:30.296165 23434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a646a13b898747b18afac954bfe0ce44" candidate_uuid: "c84391626820478bbb21215c50fb899a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af047fc88d734af0b2b6aad65d01bdb0"
I20260619 16:58:30.296221 23574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a646a13b898747b18afac954bfe0ce44" candidate_uuid: "c84391626820478bbb21215c50fb899a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17be09a87944e6f829f6198f4f21b8c"
I20260619 16:58:30.296247 23434 raft_consensus.cc:3037] T a646a13b898747b18afac954bfe0ce44 P af047fc88d734af0b2b6aad65d01bdb0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:30.296316 23574 raft_consensus.cc:3037] T a646a13b898747b18afac954bfe0ce44 P b17be09a87944e6f829f6198f4f21b8c [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:30.297008 23434 raft_consensus.cc:2454] T a646a13b898747b18afac954bfe0ce44 P af047fc88d734af0b2b6aad65d01bdb0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c84391626820478bbb21215c50fb899a in term 1.
I20260619 16:58:30.297040 23574 raft_consensus.cc:2454] T a646a13b898747b18afac954bfe0ce44 P b17be09a87944e6f829f6198f4f21b8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c84391626820478bbb21215c50fb899a in term 1.
I20260619 16:58:30.297613 23227 leader_election.cc:302] T a646a13b898747b18afac954bfe0ce44 P c84391626820478bbb21215c50fb899a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af047fc88d734af0b2b6aad65d01bdb0, c84391626820478bbb21215c50fb899a; no voters:
I20260619 16:58:30.297719 23651 raft_consensus.cc:2781] T a646a13b898747b18afac954bfe0ce44 P c84391626820478bbb21215c50fb899a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:30.297911 23651 raft_consensus.cc:686] T a646a13b898747b18afac954bfe0ce44 P c84391626820478bbb21215c50fb899a [term 1 LEADER]: Becoming Leader. State: Replica: c84391626820478bbb21215c50fb899a, State: Running, Role: LEADER
I20260619 16:58:30.298027 23651 consensus_queue.cc:237] T a646a13b898747b18afac954bfe0ce44 P c84391626820478bbb21215c50fb899a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:30.298884 23137 catalog_manager.cc:5495] T a646a13b898747b18afac954bfe0ce44 P c84391626820478bbb21215c50fb899a reported cstate change: term changed from 0 to 1, leader changed from <none> to c84391626820478bbb21215c50fb899a (127.20.231.129). New cstate: current_term: 1 leader_uuid: "c84391626820478bbb21215c50fb899a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } health_report { overall_health: UNKNOWN } } }
Jun 19 16:58:30 dist-test-slave-vnbj krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888309, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:30.335261 23660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.327745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:45859 (local address 127.0.0.1:35396)
0619 16:58:30.327927 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:30.327938 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:30.328005 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:30.328222 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:30.328223 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:30.328526 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:30.328573 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.328575 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.329255 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.329256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.329859 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.329861 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.330292 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.330293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.330345 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.331007 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:30.331012 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:30.332538 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:30.333990 (+ 1452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:30.333997 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:30.333998 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:30.334205 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:30.334446 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:30.334447 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:30.334447 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:30.334503 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:30.334946 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:30.334952 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:30.334997 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 16:58:30.335065 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 16:58:30.335174 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":81,"threads_started":1}
I20260619 16:58:30.335330 23186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.327846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35396 (local address 127.20.231.190:45859)
0619 16:58:30.327938 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:30.327940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:30.328004 (+ 64us) server_negotiation.cc:407] Connection header received
0619 16:58:30.328068 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:30.328069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:30.328078 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:30.328170 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:30.328661 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.329182 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.329944 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.330165 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.330247 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.332632 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:30.332634 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:30.332634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:30.332640 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:30.333904 (+ 1264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:30.334307 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:30.334308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:30.334309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:30.334372 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:30.334591 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:30.334591 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:30.334592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:30.334713 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:30.334832 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:30.335106 (+ 274us) server_negotiation.cc:299] Negotiation successful
0619 16:58:30.335246 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":63}
I20260619 16:58:30.337442 23137 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:35396:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 16:58:30.337618 23137 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:58:30.339330 23272 tablet_service.cc:1467] Processing CreateTablet for tablet c7d654f62c9a4e9c952c859fb8b3ed7a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=03969c32eb7c4d94be018aa6443e9bae]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 16:58:30.339309 23554 tablet_service.cc:1467] Processing CreateTablet for tablet c7d654f62c9a4e9c952c859fb8b3ed7a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=03969c32eb7c4d94be018aa6443e9bae]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 16:58:30.339352 23414 tablet_service.cc:1467] Processing CreateTablet for tablet c7d654f62c9a4e9c952c859fb8b3ed7a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=03969c32eb7c4d94be018aa6443e9bae]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 16:58:30.339519 23554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7d654f62c9a4e9c952c859fb8b3ed7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:30.339519 23414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7d654f62c9a4e9c952c859fb8b3ed7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:30.339519 23272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c7d654f62c9a4e9c952c859fb8b3ed7a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:30.341079 23644 tablet_bootstrap.cc:492] T c7d654f62c9a4e9c952c859fb8b3ed7a P b17be09a87944e6f829f6198f4f21b8c: Bootstrap starting.
I20260619 16:58:30.341102 23645 tablet_bootstrap.cc:492] T c7d654f62c9a4e9c952c859fb8b3ed7a P c84391626820478bbb21215c50fb899a: Bootstrap starting.
I20260619 16:58:30.341565 23645 tablet_bootstrap.cc:654] T c7d654f62c9a4e9c952c859fb8b3ed7a P c84391626820478bbb21215c50fb899a: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:30.341625 23644 tablet_bootstrap.cc:654] T c7d654f62c9a4e9c952c859fb8b3ed7a P b17be09a87944e6f829f6198f4f21b8c: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:30.341917 23643 tablet_bootstrap.cc:492] T c7d654f62c9a4e9c952c859fb8b3ed7a P af047fc88d734af0b2b6aad65d01bdb0: Bootstrap starting.
I20260619 16:58:30.342283 23645 tablet_bootstrap.cc:492] T c7d654f62c9a4e9c952c859fb8b3ed7a P c84391626820478bbb21215c50fb899a: No bootstrap required, opened a new log
I20260619 16:58:30.342324 23645 ts_tablet_manager.cc:1397] T c7d654f62c9a4e9c952c859fb8b3ed7a P c84391626820478bbb21215c50fb899a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:30.342497 23645 raft_consensus.cc:348] T c7d654f62c9a4e9c952c859fb8b3ed7a P c84391626820478bbb21215c50fb899a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:30.342597 23645 raft_consensus.cc:374] T c7d654f62c9a4e9c952c859fb8b3ed7a P c84391626820478bbb21215c50fb899a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:30.342645 23645 raft_consensus.cc:729] T c7d654f62c9a4e9c952c859fb8b3ed7a P c84391626820478bbb21215c50fb899a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c84391626820478bbb21215c50fb899a, State: Initialized, Role: FOLLOWER
I20260619 16:58:30.342692 23643 tablet_bootstrap.cc:654] T c7d654f62c9a4e9c952c859fb8b3ed7a P af047fc88d734af0b2b6aad65d01bdb0: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:30.342708 23645 consensus_queue.cc:260] T c7d654f62c9a4e9c952c859fb8b3ed7a P c84391626820478bbb21215c50fb899a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:30.342824 23645 ts_tablet_manager.cc:1428] T c7d654f62c9a4e9c952c859fb8b3ed7a P c84391626820478bbb21215c50fb899a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:30.342908 23651 tablet_replica.cc:439] T c7d654f62c9a4e9c952c859fb8b3ed7a P c84391626820478bbb21215c50fb899a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } } }
I20260619 16:58:30.343009 23651 tablet_replica.cc:442] T c7d654f62c9a4e9c952c859fb8b3ed7a P c84391626820478bbb21215c50fb899a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:58:30.343080 23644 tablet_bootstrap.cc:492] T c7d654f62c9a4e9c952c859fb8b3ed7a P b17be09a87944e6f829f6198f4f21b8c: No bootstrap required, opened a new log
I20260619 16:58:30.343128 23644 ts_tablet_manager.cc:1397] T c7d654f62c9a4e9c952c859fb8b3ed7a P b17be09a87944e6f829f6198f4f21b8c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:58:30.343302 23644 raft_consensus.cc:348] T c7d654f62c9a4e9c952c859fb8b3ed7a P b17be09a87944e6f829f6198f4f21b8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:30.343384 23644 raft_consensus.cc:374] T c7d654f62c9a4e9c952c859fb8b3ed7a P b17be09a87944e6f829f6198f4f21b8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:30.343407 23644 raft_consensus.cc:729] T c7d654f62c9a4e9c952c859fb8b3ed7a P b17be09a87944e6f829f6198f4f21b8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b17be09a87944e6f829f6198f4f21b8c, State: Initialized, Role: FOLLOWER
I20260619 16:58:30.343473 23644 consensus_queue.cc:260] T c7d654f62c9a4e9c952c859fb8b3ed7a P b17be09a87944e6f829f6198f4f21b8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:30.343580 23644 ts_tablet_manager.cc:1428] T c7d654f62c9a4e9c952c859fb8b3ed7a P b17be09a87944e6f829f6198f4f21b8c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 16:58:30.343705 23643 tablet_bootstrap.cc:492] T c7d654f62c9a4e9c952c859fb8b3ed7a P af047fc88d734af0b2b6aad65d01bdb0: No bootstrap required, opened a new log
I20260619 16:58:30.343763 23643 ts_tablet_manager.cc:1397] T c7d654f62c9a4e9c952c859fb8b3ed7a P af047fc88d734af0b2b6aad65d01bdb0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 16:58:30.343796 23650 tablet_replica.cc:439] T c7d654f62c9a4e9c952c859fb8b3ed7a P b17be09a87944e6f829f6198f4f21b8c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } } }
I20260619 16:58:30.343879 23643 raft_consensus.cc:348] T c7d654f62c9a4e9c952c859fb8b3ed7a P af047fc88d734af0b2b6aad65d01bdb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:30.343924 23643 raft_consensus.cc:374] T c7d654f62c9a4e9c952c859fb8b3ed7a P af047fc88d734af0b2b6aad65d01bdb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:30.343937 23643 raft_consensus.cc:729] T c7d654f62c9a4e9c952c859fb8b3ed7a P af047fc88d734af0b2b6aad65d01bdb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af047fc88d734af0b2b6aad65d01bdb0, State: Initialized, Role: FOLLOWER
I20260619 16:58:30.343923 23650 tablet_replica.cc:442] T c7d654f62c9a4e9c952c859fb8b3ed7a P b17be09a87944e6f829f6198f4f21b8c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:58:30.344002 23643 consensus_queue.cc:260] T c7d654f62c9a4e9c952c859fb8b3ed7a P af047fc88d734af0b2b6aad65d01bdb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:30.344084 23643 ts_tablet_manager.cc:1428] T c7d654f62c9a4e9c952c859fb8b3ed7a P af047fc88d734af0b2b6aad65d01bdb0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:30.344193 23649 tablet_replica.cc:439] T c7d654f62c9a4e9c952c859fb8b3ed7a P af047fc88d734af0b2b6aad65d01bdb0: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } } }
I20260619 16:58:30.344290 23649 tablet_replica.cc:442] T c7d654f62c9a4e9c952c859fb8b3ed7a P af047fc88d734af0b2b6aad65d01bdb0: This TxnStatusTablet replica's current role is: FOLLOWER
W20260619 16:58:30.375033 23341 tablet.cc:2369] T a646a13b898747b18afac954bfe0ce44 P c84391626820478bbb21215c50fb899a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 16:58:30.455595 23621 tablet.cc:2369] T a646a13b898747b18afac954bfe0ce44 P b17be09a87944e6f829f6198f4f21b8c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:58:30.469583 23649 raft_consensus.cc:482] T c7d654f62c9a4e9c952c859fb8b3ed7a P af047fc88d734af0b2b6aad65d01bdb0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:58:30.469722 23649 raft_consensus.cc:504] T c7d654f62c9a4e9c952c859fb8b3ed7a P af047fc88d734af0b2b6aad65d01bdb0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:30.470196 23649 leader_election.cc:288] T c7d654f62c9a4e9c952c859fb8b3ed7a P af047fc88d734af0b2b6aad65d01bdb0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c84391626820478bbb21215c50fb899a (127.20.231.129:35139), b17be09a87944e6f829f6198f4f21b8c (127.20.231.131:34207)
I20260619 16:58:30.473331 23373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.470375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:35139 (local address 127.20.231.130:39767)
0619 16:58:30.470451 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:30.470464 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:30.470514 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:30.470782 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:30.470783 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:30.470785 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:30.470884 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.470887 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.471673 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.471674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.472440 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.472442 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.473075 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.473076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.473165 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.473168 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:58:30.473220 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 16:58:30.473275 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260619 16:58:30.473407 23640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.470448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:39767 (local address 127.20.231.129:35139)
0619 16:58:30.470502 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:30.470504 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:30.470518 (+ 14us) server_negotiation.cc:407] Connection header received
0619 16:58:30.470597 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:30.470598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:30.470700 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:30.470765 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:30.470976 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.471581 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.472535 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.472994 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.473055 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.473062 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:30.473286 (+ 224us) server_negotiation.cc:299] Negotiation successful
0619 16:58:30.473335 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 16:58:30.473500 23664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.470380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:34207 (local address 127.20.231.130:60419)
0619 16:58:30.470645 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:30.470657 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:30.470730 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:30.470977 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:30.470978 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:30.470981 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:30.471058 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.471060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.471715 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.471717 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.472593 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.472595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.473187 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.473188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.473294 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.473298 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:58:30.473366 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 16:58:30.473403 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":65,"thread_start_us":120,"threads_started":1}
I20260619 16:58:30.473526 23641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.470472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:60419 (local address 127.20.231.131:34207)
0619 16:58:30.470515 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:30.470518 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:30.470801 (+ 283us) server_negotiation.cc:407] Connection header received
0619 16:58:30.470831 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:30.470832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:30.470870 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:30.470933 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:30.471150 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.471600 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.472690 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.473054 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.473128 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.473136 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:30.473395 (+ 259us) server_negotiation.cc:299] Negotiation successful
0619 16:58:30.473437 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 16:58:30.473719 23574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7d654f62c9a4e9c952c859fb8b3ed7a" candidate_uuid: "af047fc88d734af0b2b6aad65d01bdb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17be09a87944e6f829f6198f4f21b8c" is_pre_election: true
I20260619 16:58:30.473836 23574 raft_consensus.cc:2454] T c7d654f62c9a4e9c952c859fb8b3ed7a P b17be09a87944e6f829f6198f4f21b8c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af047fc88d734af0b2b6aad65d01bdb0 in term 0.
I20260619 16:58:30.473896 23290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7d654f62c9a4e9c952c859fb8b3ed7a" candidate_uuid: "af047fc88d734af0b2b6aad65d01bdb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c84391626820478bbb21215c50fb899a" is_pre_election: true
I20260619 16:58:30.474002 23366 leader_election.cc:302] T c7d654f62c9a4e9c952c859fb8b3ed7a P af047fc88d734af0b2b6aad65d01bdb0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af047fc88d734af0b2b6aad65d01bdb0, b17be09a87944e6f829f6198f4f21b8c; no voters:
I20260619 16:58:30.474046 23290 raft_consensus.cc:2454] T c7d654f62c9a4e9c952c859fb8b3ed7a P c84391626820478bbb21215c50fb899a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af047fc88d734af0b2b6aad65d01bdb0 in term 0.
I20260619 16:58:30.474169 23649 raft_consensus.cc:2781] T c7d654f62c9a4e9c952c859fb8b3ed7a P af047fc88d734af0b2b6aad65d01bdb0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:58:30.474217 23649 raft_consensus.cc:482] T c7d654f62c9a4e9c952c859fb8b3ed7a P af047fc88d734af0b2b6aad65d01bdb0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:58:30.474237 23649 raft_consensus.cc:3037] T c7d654f62c9a4e9c952c859fb8b3ed7a P af047fc88d734af0b2b6aad65d01bdb0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:30.475090 23649 raft_consensus.cc:504] T c7d654f62c9a4e9c952c859fb8b3ed7a P af047fc88d734af0b2b6aad65d01bdb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:30.475273 23649 leader_election.cc:288] T c7d654f62c9a4e9c952c859fb8b3ed7a P af047fc88d734af0b2b6aad65d01bdb0 [CANDIDATE]: Term 1 election: Requested vote from peers c84391626820478bbb21215c50fb899a (127.20.231.129:35139), b17be09a87944e6f829f6198f4f21b8c (127.20.231.131:34207)
I20260619 16:58:30.475394 23290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7d654f62c9a4e9c952c859fb8b3ed7a" candidate_uuid: "af047fc88d734af0b2b6aad65d01bdb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c84391626820478bbb21215c50fb899a"
I20260619 16:58:30.475497 23290 raft_consensus.cc:3037] T c7d654f62c9a4e9c952c859fb8b3ed7a P c84391626820478bbb21215c50fb899a [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:30.475515 23574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c7d654f62c9a4e9c952c859fb8b3ed7a" candidate_uuid: "af047fc88d734af0b2b6aad65d01bdb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b17be09a87944e6f829f6198f4f21b8c"
I20260619 16:58:30.475592 23574 raft_consensus.cc:3037] T c7d654f62c9a4e9c952c859fb8b3ed7a P b17be09a87944e6f829f6198f4f21b8c [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:30.476312 23574 raft_consensus.cc:2454] T c7d654f62c9a4e9c952c859fb8b3ed7a P b17be09a87944e6f829f6198f4f21b8c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af047fc88d734af0b2b6aad65d01bdb0 in term 1.
I20260619 16:58:30.476312 23290 raft_consensus.cc:2454] T c7d654f62c9a4e9c952c859fb8b3ed7a P c84391626820478bbb21215c50fb899a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af047fc88d734af0b2b6aad65d01bdb0 in term 1.
I20260619 16:58:30.476512 23366 leader_election.cc:302] T c7d654f62c9a4e9c952c859fb8b3ed7a P af047fc88d734af0b2b6aad65d01bdb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af047fc88d734af0b2b6aad65d01bdb0, b17be09a87944e6f829f6198f4f21b8c; no voters:
I20260619 16:58:30.476616 23649 raft_consensus.cc:2781] T c7d654f62c9a4e9c952c859fb8b3ed7a P af047fc88d734af0b2b6aad65d01bdb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:30.476768 23649 raft_consensus.cc:686] T c7d654f62c9a4e9c952c859fb8b3ed7a P af047fc88d734af0b2b6aad65d01bdb0 [term 1 LEADER]: Becoming Leader. State: Replica: af047fc88d734af0b2b6aad65d01bdb0, State: Running, Role: LEADER
I20260619 16:58:30.476869 23649 consensus_queue.cc:237] T c7d654f62c9a4e9c952c859fb8b3ed7a P af047fc88d734af0b2b6aad65d01bdb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:30.477317 23665 tablet_replica.cc:439] T c7d654f62c9a4e9c952c859fb8b3ed7a P af047fc88d734af0b2b6aad65d01bdb0: TxnStatusTablet state changed. Reason: New leader af047fc88d734af0b2b6aad65d01bdb0. Latest consensus state: current_term: 1 leader_uuid: "af047fc88d734af0b2b6aad65d01bdb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } } }
I20260619 16:58:30.477412 23665 tablet_replica.cc:442] T c7d654f62c9a4e9c952c859fb8b3ed7a P af047fc88d734af0b2b6aad65d01bdb0: This TxnStatusTablet replica's current role is: LEADER
I20260619 16:58:30.477694 23667 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 16:58:30.477820 23667 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 16:58:30.477769 23136 catalog_manager.cc:5495] T c7d654f62c9a4e9c952c859fb8b3ed7a P af047fc88d734af0b2b6aad65d01bdb0 reported cstate change: term changed from 0 to 1, leader changed from <none> to af047fc88d734af0b2b6aad65d01bdb0 (127.20.231.130). New cstate: current_term: 1 leader_uuid: "af047fc88d734af0b2b6aad65d01bdb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } health_report { overall_health: UNKNOWN } } }
I20260619 16:58:30.490073 23642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.487563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56228 (local address 127.20.231.130:34463)
0619 16:58:30.487611 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:30.487612 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:30.487622 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:58:30.487710 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:30.487711 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:30.487731 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:30.487794 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:58:30.487926 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.488436 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.489222 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.489436 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.489513 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.489705 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:58:30.489810 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:58:30.489899 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:30.489951 (+ 52us) server_negotiation.cc:299] Negotiation successful
0619 16:58:30.489972 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 16:58:30.490070 23660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.487492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:34463 (local address 127.0.0.1:56228)
0619 16:58:30.487563 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:30.487575 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:30.487630 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:30.487804 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:30.487805 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:30.487806 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 16:58:30.487858 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.487860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.488522 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.488523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.489133 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.489135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.489511 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.489512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.489578 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.489599 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 16:58:30.489879 (+ 280us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 16:58:30.489888 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 16:58:30.489946 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 16:58:30.489997 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 16:58:30.492185 23574 raft_consensus.cc:1261] T c7d654f62c9a4e9c952c859fb8b3ed7a P b17be09a87944e6f829f6198f4f21b8c [term 1 FOLLOWER]: Refusing update from remote peer af047fc88d734af0b2b6aad65d01bdb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:30.492175 23290 raft_consensus.cc:1261] T c7d654f62c9a4e9c952c859fb8b3ed7a P c84391626820478bbb21215c50fb899a [term 1 FOLLOWER]: Refusing update from remote peer af047fc88d734af0b2b6aad65d01bdb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:30.492496 23665 consensus_queue.cc:1035] T c7d654f62c9a4e9c952c859fb8b3ed7a P af047fc88d734af0b2b6aad65d01bdb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:30.492645 23649 consensus_queue.cc:1035] T c7d654f62c9a4e9c952c859fb8b3ed7a P af047fc88d734af0b2b6aad65d01bdb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:30.493825 23650 tablet_replica.cc:439] T c7d654f62c9a4e9c952c859fb8b3ed7a P b17be09a87944e6f829f6198f4f21b8c: TxnStatusTablet state changed. Reason: New leader af047fc88d734af0b2b6aad65d01bdb0. Latest consensus state: current_term: 1 leader_uuid: "af047fc88d734af0b2b6aad65d01bdb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } } }
I20260619 16:58:30.493968 23650 tablet_replica.cc:442] T c7d654f62c9a4e9c952c859fb8b3ed7a P b17be09a87944e6f829f6198f4f21b8c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:58:30.494288 23651 tablet_replica.cc:439] T c7d654f62c9a4e9c952c859fb8b3ed7a P c84391626820478bbb21215c50fb899a: TxnStatusTablet state changed. Reason: New leader af047fc88d734af0b2b6aad65d01bdb0. Latest consensus state: current_term: 1 leader_uuid: "af047fc88d734af0b2b6aad65d01bdb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } } }
I20260619 16:58:30.494383 23651 tablet_replica.cc:442] T c7d654f62c9a4e9c952c859fb8b3ed7a P c84391626820478bbb21215c50fb899a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:58:30.495618 23665 tablet_replica.cc:439] T c7d654f62c9a4e9c952c859fb8b3ed7a P af047fc88d734af0b2b6aad65d01bdb0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af047fc88d734af0b2b6aad65d01bdb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } } }
I20260619 16:58:30.495724 23665 tablet_replica.cc:442] T c7d654f62c9a4e9c952c859fb8b3ed7a P af047fc88d734af0b2b6aad65d01bdb0: This TxnStatusTablet replica's current role is: LEADER
I20260619 16:58:30.495834 23665 tablet_replica.cc:439] T c7d654f62c9a4e9c952c859fb8b3ed7a P af047fc88d734af0b2b6aad65d01bdb0: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "af047fc88d734af0b2b6aad65d01bdb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } } }
I20260619 16:58:30.495831 23651 tablet_replica.cc:439] T c7d654f62c9a4e9c952c859fb8b3ed7a P c84391626820478bbb21215c50fb899a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af047fc88d734af0b2b6aad65d01bdb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } } }
I20260619 16:58:30.495888 23665 tablet_replica.cc:442] T c7d654f62c9a4e9c952c859fb8b3ed7a P af047fc88d734af0b2b6aad65d01bdb0: This TxnStatusTablet replica's current role is: LEADER
I20260619 16:58:30.495908 23651 tablet_replica.cc:442] T c7d654f62c9a4e9c952c859fb8b3ed7a P c84391626820478bbb21215c50fb899a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:58:30.496371 23650 tablet_replica.cc:439] T c7d654f62c9a4e9c952c859fb8b3ed7a P b17be09a87944e6f829f6198f4f21b8c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "af047fc88d734af0b2b6aad65d01bdb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } } }
I20260619 16:58:30.496452 23650 tablet_replica.cc:442] T c7d654f62c9a4e9c952c859fb8b3ed7a P b17be09a87944e6f829f6198f4f21b8c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:58:30.496667 23672 mvcc.cc:205] Tried to move back new op lower bound from 7298614519773057024 to 7298614519713759232. Current Snapshot: MvccSnapshot[applied={T|T < 7298614519773057024}]
I20260619 16:58:30.496704 23671 mvcc.cc:205] Tried to move back new op lower bound from 7298614519773057024 to 7298614519713759232. Current Snapshot: MvccSnapshot[applied={T|T < 7298614519773057024}]
I20260619 16:58:30.502039 23640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.499384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47074 (local address 127.20.231.129:35139)
0619 16:58:30.499424 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:30.499426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:30.499435 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:58:30.499511 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:30.499511 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:30.499526 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:30.499580 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:58:30.499710 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.500181 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.501342 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.501523 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.501578 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.501743 (+ 165us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:58:30.501844 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:58:30.501906 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:30.501968 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 16:58:30.501985 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 16:58:30.507434 23681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.504623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:34463 (local address 127.20.231.129:54625)
0619 16:58:30.504808 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:30.504819 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:30.504882 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:30.504996 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:30.504996 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:30.504997 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 16:58:30.505042 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.505043 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.505995 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.505996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.506578 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.506579 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.507056 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.507057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.507105 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.507117 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 16:58:30.507301 (+ 184us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 16:58:30.507302 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 16:58:30.507338 (+ 36us) client_negotiation.cc:240] Negotiation successful
0619 16:58:30.507372 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":120,"threads_started":1}
I20260619 16:58:30.507441 23642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.504713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:54625 (local address 127.20.231.130:34463)
0619 16:58:30.504754 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:30.504756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:30.504888 (+ 132us) server_negotiation.cc:407] Connection header received
0619 16:58:30.504920 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:30.504921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:30.504931 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:30.504993 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:58:30.505091 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.505895 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.506669 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.506984 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.507058 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.507207 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:58:30.507241 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:58:30.507296 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:30.507342 (+ 46us) server_negotiation.cc:299] Negotiation successful
0619 16:58:30.507362 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 16:58:30.510238 23574 raft_consensus.cc:1261] T a646a13b898747b18afac954bfe0ce44 P b17be09a87944e6f829f6198f4f21b8c [term 1 FOLLOWER]: Refusing update from remote peer c84391626820478bbb21215c50fb899a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:30.510273 23434 raft_consensus.cc:1261] T a646a13b898747b18afac954bfe0ce44 P af047fc88d734af0b2b6aad65d01bdb0 [term 1 FOLLOWER]: Refusing update from remote peer c84391626820478bbb21215c50fb899a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:30.510533 23651 consensus_queue.cc:1035] T a646a13b898747b18afac954bfe0ce44 P c84391626820478bbb21215c50fb899a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:30.510700 23653 consensus_queue.cc:1035] T a646a13b898747b18afac954bfe0ce44 P c84391626820478bbb21215c50fb899a [LEADER]: Connected to new peer: Peer: permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:30.519394 23486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.516662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:35139 (local address 127.20.231.130:33311)
0619 16:58:30.516738 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:30.516752 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:30.516814 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:30.516935 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:30.516936 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:30.516937 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 16:58:30.517000 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.517001 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.517645 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.517646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.518510 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.518513 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.518932 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.518933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.518999 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.519021 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 16:58:30.519232 (+ 211us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 16:58:30.519234 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 16:58:30.519287 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 16:58:30.519328 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 16:58:30.519388 23640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.516721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:33311 (local address 127.20.231.129:35139)
0619 16:58:30.516767 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:30.516769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:30.516793 (+ 24us) server_negotiation.cc:407] Connection header received
0619 16:58:30.516856 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:30.516856 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:30.516864 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:30.516931 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:58:30.517076 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.517560 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.518603 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.518812 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.518932 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.519112 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:58:30.519154 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:58:30.519229 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:30.519291 (+ 62us) server_negotiation.cc:299] Negotiation successful
0619 16:58:30.519311 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 16:58:30.528416 23137 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35380:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:58:30.528537 23137 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35380:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:58:30.529322 23137 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d95249b6e2324671b047117583f5f601: Sending DeleteTablet for 3 replicas of tablet a646a13b898747b18afac954bfe0ce44
I20260619 16:58:30.529625 23414 tablet_service.cc:1514] Processing DeleteTablet for tablet a646a13b898747b18afac954bfe0ce44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:30 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:56216
I20260619 16:58:30.529783 23554 tablet_service.cc:1514] Processing DeleteTablet for tablet a646a13b898747b18afac954bfe0ce44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:30 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:37310
I20260619 16:58:30.529877 23689 tablet_replica.cc:330] T a646a13b898747b18afac954bfe0ce44 P af047fc88d734af0b2b6aad65d01bdb0: stopping tablet replica
I20260619 16:58:30.529978 23689 raft_consensus.cc:2229] T a646a13b898747b18afac954bfe0ce44 P af047fc88d734af0b2b6aad65d01bdb0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:58:30.530040 23689 raft_consensus.cc:2258] T a646a13b898747b18afac954bfe0ce44 P af047fc88d734af0b2b6aad65d01bdb0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:30.530189 23690 tablet_replica.cc:330] T a646a13b898747b18afac954bfe0ce44 P b17be09a87944e6f829f6198f4f21b8c: stopping tablet replica
I20260619 16:58:30.530292 23690 raft_consensus.cc:2229] T a646a13b898747b18afac954bfe0ce44 P b17be09a87944e6f829f6198f4f21b8c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:58:30.530352 23690 raft_consensus.cc:2258] T a646a13b898747b18afac954bfe0ce44 P b17be09a87944e6f829f6198f4f21b8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:30.530524 23689 ts_tablet_manager.cc:1905] T a646a13b898747b18afac954bfe0ce44 P af047fc88d734af0b2b6aad65d01bdb0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:30.530567 23272 tablet_service.cc:1514] Processing DeleteTablet for tablet a646a13b898747b18afac954bfe0ce44 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:30 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:47066
I20260619 16:58:30.530781 23691 tablet_replica.cc:330] T a646a13b898747b18afac954bfe0ce44 P c84391626820478bbb21215c50fb899a: stopping tablet replica
I20260619 16:58:30.530941 23691 raft_consensus.cc:2229] T a646a13b898747b18afac954bfe0ce44 P c84391626820478bbb21215c50fb899a [term 1 LEADER]: Raft consensus shutting down.
I20260619 16:58:30.530912 23137 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35380:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:58:30.531052 23137 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:58:30.531298 23691 raft_consensus.cc:2258] T a646a13b898747b18afac954bfe0ce44 P c84391626820478bbb21215c50fb899a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:30.531795 23691 ts_tablet_manager.cc:1905] T a646a13b898747b18afac954bfe0ce44 P c84391626820478bbb21215c50fb899a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:30.532254 23414 tablet_service.cc:1467] Processing CreateTablet for tablet 28474855969c4ca6bb0ee277f1323df2 (DEFAULT_TABLE table=test-table [id=e8194ee6aaf7418697ca312cc76ea3fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:30.532359 23414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28474855969c4ca6bb0ee277f1323df2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:30.532445 23554 tablet_service.cc:1467] Processing CreateTablet for tablet 28474855969c4ca6bb0ee277f1323df2 (DEFAULT_TABLE table=test-table [id=e8194ee6aaf7418697ca312cc76ea3fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:30.532549 23554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28474855969c4ca6bb0ee277f1323df2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:30.532934 23689 ts_tablet_manager.cc:1918] T a646a13b898747b18afac954bfe0ce44 P af047fc88d734af0b2b6aad65d01bdb0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 16:58:30.532994 23689 log.cc:1196] T a646a13b898747b18afac954bfe0ce44 P af047fc88d734af0b2b6aad65d01bdb0: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-1/wal/wals/a646a13b898747b18afac954bfe0ce44
I20260619 16:58:30.533275 23689 ts_tablet_manager.cc:1939] T a646a13b898747b18afac954bfe0ce44 P af047fc88d734af0b2b6aad65d01bdb0: Deleting consensus metadata
I20260619 16:58:30.533723 23123 catalog_manager.cc:4841] TS af047fc88d734af0b2b6aad65d01bdb0 (127.20.231.130:34463): tablet a646a13b898747b18afac954bfe0ce44 (table test-table [id=570b34468a514f7f9f50bd96477061c8]) successfully deleted
I20260619 16:58:30.533782 23643 tablet_bootstrap.cc:492] T 28474855969c4ca6bb0ee277f1323df2 P af047fc88d734af0b2b6aad65d01bdb0: Bootstrap starting.
I20260619 16:58:30.534183 23643 tablet_bootstrap.cc:654] T 28474855969c4ca6bb0ee277f1323df2 P af047fc88d734af0b2b6aad65d01bdb0: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:30.534262 23272 tablet_service.cc:1467] Processing CreateTablet for tablet 28474855969c4ca6bb0ee277f1323df2 (DEFAULT_TABLE table=test-table [id=e8194ee6aaf7418697ca312cc76ea3fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:30.534363 23272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 28474855969c4ca6bb0ee277f1323df2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:30.534580 23644 tablet_bootstrap.cc:492] T 28474855969c4ca6bb0ee277f1323df2 P b17be09a87944e6f829f6198f4f21b8c: Bootstrap starting.
I20260619 16:58:30.534987 23644 tablet_bootstrap.cc:654] T 28474855969c4ca6bb0ee277f1323df2 P b17be09a87944e6f829f6198f4f21b8c: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:30.535681 23645 tablet_bootstrap.cc:492] T 28474855969c4ca6bb0ee277f1323df2 P c84391626820478bbb21215c50fb899a: Bootstrap starting.
I20260619 16:58:30.536067 23645 tablet_bootstrap.cc:654] T 28474855969c4ca6bb0ee277f1323df2 P c84391626820478bbb21215c50fb899a: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:30.536898 23690 ts_tablet_manager.cc:1905] T a646a13b898747b18afac954bfe0ce44 P b17be09a87944e6f829f6198f4f21b8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:30.537065 23691 ts_tablet_manager.cc:1918] T a646a13b898747b18afac954bfe0ce44 P c84391626820478bbb21215c50fb899a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 16:58:30.537173 23691 log.cc:1196] T a646a13b898747b18afac954bfe0ce44 P c84391626820478bbb21215c50fb899a: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-0/wal/wals/a646a13b898747b18afac954bfe0ce44
I20260619 16:58:30.537479 23691 ts_tablet_manager.cc:1939] T a646a13b898747b18afac954bfe0ce44 P c84391626820478bbb21215c50fb899a: Deleting consensus metadata
I20260619 16:58:30.537904 23121 catalog_manager.cc:4841] TS c84391626820478bbb21215c50fb899a (127.20.231.129:35139): tablet a646a13b898747b18afac954bfe0ce44 (table test-table [id=570b34468a514f7f9f50bd96477061c8]) successfully deleted
I20260619 16:58:30.538384 23643 tablet_bootstrap.cc:492] T 28474855969c4ca6bb0ee277f1323df2 P af047fc88d734af0b2b6aad65d01bdb0: No bootstrap required, opened a new log
I20260619 16:58:30.538431 23643 ts_tablet_manager.cc:1397] T 28474855969c4ca6bb0ee277f1323df2 P af047fc88d734af0b2b6aad65d01bdb0: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260619 16:58:30.538569 23643 raft_consensus.cc:348] T 28474855969c4ca6bb0ee277f1323df2 P af047fc88d734af0b2b6aad65d01bdb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:30.538636 23643 raft_consensus.cc:374] T 28474855969c4ca6bb0ee277f1323df2 P af047fc88d734af0b2b6aad65d01bdb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:30.538658 23643 raft_consensus.cc:729] T 28474855969c4ca6bb0ee277f1323df2 P af047fc88d734af0b2b6aad65d01bdb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af047fc88d734af0b2b6aad65d01bdb0, State: Initialized, Role: FOLLOWER
I20260619 16:58:30.538715 23643 consensus_queue.cc:260] T 28474855969c4ca6bb0ee277f1323df2 P af047fc88d734af0b2b6aad65d01bdb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:30.538762 23690 ts_tablet_manager.cc:1918] T a646a13b898747b18afac954bfe0ce44 P b17be09a87944e6f829f6198f4f21b8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 16:58:30.538806 23643 ts_tablet_manager.cc:1428] T 28474855969c4ca6bb0ee277f1323df2 P af047fc88d734af0b2b6aad65d01bdb0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:30.538826 23690 log.cc:1196] T a646a13b898747b18afac954bfe0ce44 P b17be09a87944e6f829f6198f4f21b8c: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-2/wal/wals/a646a13b898747b18afac954bfe0ce44
I20260619 16:58:30.539089 23690 ts_tablet_manager.cc:1939] T a646a13b898747b18afac954bfe0ce44 P b17be09a87944e6f829f6198f4f21b8c: Deleting consensus metadata
I20260619 16:58:30.539541 23122 catalog_manager.cc:4841] TS b17be09a87944e6f829f6198f4f21b8c (127.20.231.131:34207): tablet a646a13b898747b18afac954bfe0ce44 (table test-table [id=570b34468a514f7f9f50bd96477061c8]) successfully deleted
I20260619 16:58:30.539997 23645 tablet_bootstrap.cc:492] T 28474855969c4ca6bb0ee277f1323df2 P c84391626820478bbb21215c50fb899a: No bootstrap required, opened a new log
I20260619 16:58:30.540046 23645 ts_tablet_manager.cc:1397] T 28474855969c4ca6bb0ee277f1323df2 P c84391626820478bbb21215c50fb899a: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260619 16:58:30.540210 23645 raft_consensus.cc:348] T 28474855969c4ca6bb0ee277f1323df2 P c84391626820478bbb21215c50fb899a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:30.540274 23645 raft_consensus.cc:374] T 28474855969c4ca6bb0ee277f1323df2 P c84391626820478bbb21215c50fb899a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:30.540294 23645 raft_consensus.cc:729] T 28474855969c4ca6bb0ee277f1323df2 P c84391626820478bbb21215c50fb899a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c84391626820478bbb21215c50fb899a, State: Initialized, Role: FOLLOWER
I20260619 16:58:30.540342 23645 consensus_queue.cc:260] T 28474855969c4ca6bb0ee277f1323df2 P c84391626820478bbb21215c50fb899a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:30.540426 23645 ts_tablet_manager.cc:1428] T 28474855969c4ca6bb0ee277f1323df2 P c84391626820478bbb21215c50fb899a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:30.543095 23644 tablet_bootstrap.cc:492] T 28474855969c4ca6bb0ee277f1323df2 P b17be09a87944e6f829f6198f4f21b8c: No bootstrap required, opened a new log
I20260619 16:58:30.543161 23644 ts_tablet_manager.cc:1397] T 28474855969c4ca6bb0ee277f1323df2 P b17be09a87944e6f829f6198f4f21b8c: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20260619 16:58:30.543305 23644 raft_consensus.cc:348] T 28474855969c4ca6bb0ee277f1323df2 P b17be09a87944e6f829f6198f4f21b8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:30.543380 23644 raft_consensus.cc:374] T 28474855969c4ca6bb0ee277f1323df2 P b17be09a87944e6f829f6198f4f21b8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:30.543401 23644 raft_consensus.cc:729] T 28474855969c4ca6bb0ee277f1323df2 P b17be09a87944e6f829f6198f4f21b8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b17be09a87944e6f829f6198f4f21b8c, State: Initialized, Role: FOLLOWER
I20260619 16:58:30.543449 23644 consensus_queue.cc:260] T 28474855969c4ca6bb0ee277f1323df2 P b17be09a87944e6f829f6198f4f21b8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:30.543535 23644 ts_tablet_manager.cc:1428] T 28474855969c4ca6bb0ee277f1323df2 P b17be09a87944e6f829f6198f4f21b8c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 16:58:30.566354 23650 raft_consensus.cc:482] T 28474855969c4ca6bb0ee277f1323df2 P b17be09a87944e6f829f6198f4f21b8c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:58:30.566473 23650 raft_consensus.cc:504] T 28474855969c4ca6bb0ee277f1323df2 P b17be09a87944e6f829f6198f4f21b8c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:30.566824 23650 leader_election.cc:288] T 28474855969c4ca6bb0ee277f1323df2 P b17be09a87944e6f829f6198f4f21b8c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c84391626820478bbb21215c50fb899a (127.20.231.129:35139), af047fc88d734af0b2b6aad65d01bdb0 (127.20.231.130:34463)
I20260619 16:58:30.570936 23513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.567028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:35139 (local address 127.20.231.131:49885)
0619 16:58:30.567138 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:30.567150 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:30.567192 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:30.567490 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:30.567491 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:30.567493 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:30.567561 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.567564 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.568568 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.568569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.569798 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.569802 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.570564 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.570565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.570687 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.570691 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:58:30.570747 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 16:58:30.570797 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260619 16:58:30.571265 23640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.567273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:49885 (local address 127.20.231.129:35139)
0619 16:58:30.567333 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:30.567335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:30.567344 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:58:30.567378 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:30.567379 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:30.567389 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:30.567439 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:30.567671 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.568460 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.569918 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.570442 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.570519 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.570530 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:30.571160 (+ 630us) server_negotiation.cc:299] Negotiation successful
0619 16:58:30.571200 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 16:58:30.571514 23290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28474855969c4ca6bb0ee277f1323df2" candidate_uuid: "b17be09a87944e6f829f6198f4f21b8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c84391626820478bbb21215c50fb899a" is_pre_election: true
I20260619 16:58:30.571623 23290 raft_consensus.cc:2454] T 28474855969c4ca6bb0ee277f1323df2 P c84391626820478bbb21215c50fb899a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b17be09a87944e6f829f6198f4f21b8c in term 0.
I20260619 16:58:30.571719 23642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.567226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:34051 (local address 127.20.231.130:34463)
0619 16:58:30.567268 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:30.567271 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:30.567470 (+ 199us) server_negotiation.cc:407] Connection header received
0619 16:58:30.567570 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:30.567571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:30.567580 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:30.567609 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:30.568145 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.568877 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.570270 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.570826 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.570916 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.570924 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:30.571596 (+ 672us) server_negotiation.cc:299] Negotiation successful
0619 16:58:30.571637 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 16:58:30.571835 23505 leader_election.cc:302] T 28474855969c4ca6bb0ee277f1323df2 P b17be09a87944e6f829f6198f4f21b8c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b17be09a87944e6f829f6198f4f21b8c, c84391626820478bbb21215c50fb899a; no voters:
I20260619 16:58:30.571969 23650 raft_consensus.cc:2781] T 28474855969c4ca6bb0ee277f1323df2 P b17be09a87944e6f829f6198f4f21b8c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:58:30.572001 23650 raft_consensus.cc:482] T 28474855969c4ca6bb0ee277f1323df2 P b17be09a87944e6f829f6198f4f21b8c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:58:30.572026 23650 raft_consensus.cc:3037] T 28474855969c4ca6bb0ee277f1323df2 P b17be09a87944e6f829f6198f4f21b8c [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:30.572211 23695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.567080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:34463 (local address 127.20.231.131:34051)
0619 16:58:30.567408 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:30.567418 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:30.567495 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:30.567671 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:30.567672 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:30.567673 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:30.568053 (+ 380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.568055 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.568983 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.568984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.570165 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.570168 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.570980 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.570980 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.571285 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.571290 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 16:58:30.572079 (+ 789us) client_negotiation.cc:240] Negotiation successful
0619 16:58:30.572127 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":56,"threads_started":1}
I20260619 16:58:30.572531 23434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28474855969c4ca6bb0ee277f1323df2" candidate_uuid: "b17be09a87944e6f829f6198f4f21b8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af047fc88d734af0b2b6aad65d01bdb0" is_pre_election: true
I20260619 16:58:30.572611 23434 raft_consensus.cc:2454] T 28474855969c4ca6bb0ee277f1323df2 P af047fc88d734af0b2b6aad65d01bdb0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b17be09a87944e6f829f6198f4f21b8c in term 0.
I20260619 16:58:30.572955 23650 raft_consensus.cc:504] T 28474855969c4ca6bb0ee277f1323df2 P b17be09a87944e6f829f6198f4f21b8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:30.573122 23650 leader_election.cc:288] T 28474855969c4ca6bb0ee277f1323df2 P b17be09a87944e6f829f6198f4f21b8c [CANDIDATE]: Term 1 election: Requested vote from peers c84391626820478bbb21215c50fb899a (127.20.231.129:35139), af047fc88d734af0b2b6aad65d01bdb0 (127.20.231.130:34463)
I20260619 16:58:30.573302 23434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28474855969c4ca6bb0ee277f1323df2" candidate_uuid: "b17be09a87944e6f829f6198f4f21b8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af047fc88d734af0b2b6aad65d01bdb0"
I20260619 16:58:30.573374 23434 raft_consensus.cc:3037] T 28474855969c4ca6bb0ee277f1323df2 P af047fc88d734af0b2b6aad65d01bdb0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:30.573526 23290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "28474855969c4ca6bb0ee277f1323df2" candidate_uuid: "b17be09a87944e6f829f6198f4f21b8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c84391626820478bbb21215c50fb899a"
I20260619 16:58:30.573602 23290 raft_consensus.cc:3037] T 28474855969c4ca6bb0ee277f1323df2 P c84391626820478bbb21215c50fb899a [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:30.574153 23434 raft_consensus.cc:2454] T 28474855969c4ca6bb0ee277f1323df2 P af047fc88d734af0b2b6aad65d01bdb0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b17be09a87944e6f829f6198f4f21b8c in term 1.
I20260619 16:58:30.574335 23507 leader_election.cc:302] T 28474855969c4ca6bb0ee277f1323df2 P b17be09a87944e6f829f6198f4f21b8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af047fc88d734af0b2b6aad65d01bdb0, b17be09a87944e6f829f6198f4f21b8c; no voters:
I20260619 16:58:30.574411 23290 raft_consensus.cc:2454] T 28474855969c4ca6bb0ee277f1323df2 P c84391626820478bbb21215c50fb899a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b17be09a87944e6f829f6198f4f21b8c in term 1.
I20260619 16:58:30.574433 23650 raft_consensus.cc:2781] T 28474855969c4ca6bb0ee277f1323df2 P b17be09a87944e6f829f6198f4f21b8c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:30.574579 23650 raft_consensus.cc:686] T 28474855969c4ca6bb0ee277f1323df2 P b17be09a87944e6f829f6198f4f21b8c [term 1 LEADER]: Becoming Leader. State: Replica: b17be09a87944e6f829f6198f4f21b8c, State: Running, Role: LEADER
I20260619 16:58:30.574656 23650 consensus_queue.cc:237] T 28474855969c4ca6bb0ee277f1323df2 P b17be09a87944e6f829f6198f4f21b8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:30.575297 23137 catalog_manager.cc:5495] T 28474855969c4ca6bb0ee277f1323df2 P b17be09a87944e6f829f6198f4f21b8c reported cstate change: term changed from 0 to 1, leader changed from <none> to b17be09a87944e6f829f6198f4f21b8c (127.20.231.131). New cstate: current_term: 1 leader_uuid: "b17be09a87944e6f829f6198f4f21b8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } health_report { overall_health: HEALTHY } } }
I20260619 16:58:30.590595 23641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.587145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37320 (local address 127.20.231.131:34207)
0619 16:58:30.587202 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:30.587205 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:30.587236 (+ 31us) server_negotiation.cc:407] Connection header received
0619 16:58:30.587273 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:30.587274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:30.587296 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:30.587380 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:58:30.587517 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.588275 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.589414 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.589709 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.589777 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.589979 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:58:30.590136 (+ 157us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:58:30.590213 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:30.590470 (+ 257us) server_negotiation.cc:299] Negotiation successful
0619 16:58:30.590495 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 16:58:30.596789 23626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.593514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:34463 (local address 127.20.231.131:48705)
0619 16:58:30.593579 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:30.593591 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:30.593652 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:30.593811 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:30.593812 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:30.593813 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 16:58:30.593874 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.593875 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.594885 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.594887 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.595782 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.595785 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.596303 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.596304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.596370 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.596400 (+ 30us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 16:58:30.596603 (+ 203us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 16:58:30.596605 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 16:58:30.596647 (+ 42us) client_negotiation.cc:240] Negotiation successful
0619 16:58:30.596697 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260619 16:58:30.597358 23642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.593583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:48705 (local address 127.20.231.130:34463)
0619 16:58:30.593634 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:30.593636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:30.593646 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:58:30.593708 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:30.593709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:30.593723 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:30.593770 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:58:30.593944 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.594730 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.595888 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.596196 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.596267 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.596496 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:58:30.596531 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:58:30.596580 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:30.597240 (+ 660us) server_negotiation.cc:299] Negotiation successful
0619 16:58:30.597266 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 16:58:30.600102 23434 raft_consensus.cc:1261] T 28474855969c4ca6bb0ee277f1323df2 P af047fc88d734af0b2b6aad65d01bdb0 [term 1 FOLLOWER]: Refusing update from remote peer b17be09a87944e6f829f6198f4f21b8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:30.600111 23290 raft_consensus.cc:1261] T 28474855969c4ca6bb0ee277f1323df2 P c84391626820478bbb21215c50fb899a [term 1 FOLLOWER]: Refusing update from remote peer b17be09a87944e6f829f6198f4f21b8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:30.600369 23696 consensus_queue.cc:1035] T 28474855969c4ca6bb0ee277f1323df2 P b17be09a87944e6f829f6198f4f21b8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:30.600483 23650 consensus_queue.cc:1035] T 28474855969c4ca6bb0ee277f1323df2 P b17be09a87944e6f829f6198f4f21b8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:30.602985 23670 mvcc.cc:205] Tried to move back new op lower bound from 7298614520216395776 to 7298614520114380800. Current Snapshot: MvccSnapshot[applied={T|T < 7298614520216395776}]
I20260619 16:58:30.612211 23641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.608798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:58727 (local address 127.20.231.131:34207)
0619 16:58:30.608838 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:30.608840 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:30.608847 (+ 7us) server_negotiation.cc:407] Connection header received
0619 16:58:30.608883 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:30.608884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:30.608894 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:30.608943 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:58:30.609116 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.609941 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.611179 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.611573 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.611644 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.611885 (+ 241us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:58:30.611922 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:58:30.611996 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:30.612084 (+ 88us) server_negotiation.cc:299] Negotiation successful
0619 16:58:30.612111 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 16:58:30.612238 23486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:30.608571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:34207 (local address 127.20.231.130:58727)
0619 16:58:30.608668 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:30.608680 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:30.608722 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:30.608989 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:30.608990 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:30.608991 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 16:58:30.609046 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.609048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.610047 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.610048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.611072 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:30.611076 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:30.611673 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:30.611674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:30.611788 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:30.611795 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 16:58:30.612016 (+ 221us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 16:58:30.612018 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 16:58:30.612071 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 16:58:30.612126 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
W20260619 16:58:30.612583 23672 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 16:58:32.142499 23137 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35380:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:58:32.142617 23137 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35380:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:58:32.143574 23137 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d95249b6e2324671b047117583f5f601: Sending DeleteTablet for 3 replicas of tablet 28474855969c4ca6bb0ee277f1323df2
I20260619 16:58:32.143853 23272 tablet_service.cc:1514] Processing DeleteTablet for tablet 28474855969c4ca6bb0ee277f1323df2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:32 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:47066
I20260619 16:58:32.143931 23414 tablet_service.cc:1514] Processing DeleteTablet for tablet 28474855969c4ca6bb0ee277f1323df2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:32 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:56216
I20260619 16:58:32.144140 23554 tablet_service.cc:1514] Processing DeleteTablet for tablet 28474855969c4ca6bb0ee277f1323df2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:32 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:37310
I20260619 16:58:32.144392 23725 tablet_replica.cc:330] T 28474855969c4ca6bb0ee277f1323df2 P af047fc88d734af0b2b6aad65d01bdb0: stopping tablet replica
I20260619 16:58:32.144456 23725 raft_consensus.cc:2229] T 28474855969c4ca6bb0ee277f1323df2 P af047fc88d734af0b2b6aad65d01bdb0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:58:32.144498 23725 raft_consensus.cc:2258] T 28474855969c4ca6bb0ee277f1323df2 P af047fc88d734af0b2b6aad65d01bdb0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:32.144755 23725 ts_tablet_manager.cc:1905] T 28474855969c4ca6bb0ee277f1323df2 P af047fc88d734af0b2b6aad65d01bdb0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:32.146734 23725 ts_tablet_manager.cc:1918] T 28474855969c4ca6bb0ee277f1323df2 P af047fc88d734af0b2b6aad65d01bdb0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 16:58:32.146802 23725 log.cc:1196] T 28474855969c4ca6bb0ee277f1323df2 P af047fc88d734af0b2b6aad65d01bdb0: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-1/wal/wals/28474855969c4ca6bb0ee277f1323df2
I20260619 16:58:32.146919 23724 tablet_replica.cc:330] T 28474855969c4ca6bb0ee277f1323df2 P c84391626820478bbb21215c50fb899a: stopping tablet replica
I20260619 16:58:32.146989 23724 raft_consensus.cc:2229] T 28474855969c4ca6bb0ee277f1323df2 P c84391626820478bbb21215c50fb899a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:58:32.147040 23724 raft_consensus.cc:2258] T 28474855969c4ca6bb0ee277f1323df2 P c84391626820478bbb21215c50fb899a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:32.147069 23725 ts_tablet_manager.cc:1939] T 28474855969c4ca6bb0ee277f1323df2 P af047fc88d734af0b2b6aad65d01bdb0: Deleting consensus metadata
I20260619 16:58:32.147132 23726 tablet_replica.cc:330] T 28474855969c4ca6bb0ee277f1323df2 P b17be09a87944e6f829f6198f4f21b8c: stopping tablet replica
I20260619 16:58:32.147194 23726 raft_consensus.cc:2229] T 28474855969c4ca6bb0ee277f1323df2 P b17be09a87944e6f829f6198f4f21b8c [term 1 LEADER]: Raft consensus shutting down.
I20260619 16:58:32.147294 23726 raft_consensus.cc:2258] T 28474855969c4ca6bb0ee277f1323df2 P b17be09a87944e6f829f6198f4f21b8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:32.147560 23123 catalog_manager.cc:4841] TS af047fc88d734af0b2b6aad65d01bdb0 (127.20.231.130:34463): tablet 28474855969c4ca6bb0ee277f1323df2 (table test-table [id=e8194ee6aaf7418697ca312cc76ea3fb]) successfully deleted
I20260619 16:58:32.147634 23724 ts_tablet_manager.cc:1905] T 28474855969c4ca6bb0ee277f1323df2 P c84391626820478bbb21215c50fb899a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:32.149129 23724 ts_tablet_manager.cc:1918] T 28474855969c4ca6bb0ee277f1323df2 P c84391626820478bbb21215c50fb899a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 16:58:32.149209 23724 log.cc:1196] T 28474855969c4ca6bb0ee277f1323df2 P c84391626820478bbb21215c50fb899a: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-0/wal/wals/28474855969c4ca6bb0ee277f1323df2
I20260619 16:58:32.149458 23724 ts_tablet_manager.cc:1939] T 28474855969c4ca6bb0ee277f1323df2 P c84391626820478bbb21215c50fb899a: Deleting consensus metadata
I20260619 16:58:32.149758 23726 ts_tablet_manager.cc:1905] T 28474855969c4ca6bb0ee277f1323df2 P b17be09a87944e6f829f6198f4f21b8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:32.150110 23121 catalog_manager.cc:4841] TS c84391626820478bbb21215c50fb899a (127.20.231.129:35139): tablet 28474855969c4ca6bb0ee277f1323df2 (table test-table [id=e8194ee6aaf7418697ca312cc76ea3fb]) successfully deleted
I20260619 16:58:32.151726 23726 ts_tablet_manager.cc:1918] T 28474855969c4ca6bb0ee277f1323df2 P b17be09a87944e6f829f6198f4f21b8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 16:58:32.151805 23726 log.cc:1196] T 28474855969c4ca6bb0ee277f1323df2 P b17be09a87944e6f829f6198f4f21b8c: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-2/wal/wals/28474855969c4ca6bb0ee277f1323df2
I20260619 16:58:32.152019 23726 ts_tablet_manager.cc:1939] T 28474855969c4ca6bb0ee277f1323df2 P b17be09a87944e6f829f6198f4f21b8c: Deleting consensus metadata
I20260619 16:58:32.152403 23122 catalog_manager.cc:4841] TS b17be09a87944e6f829f6198f4f21b8c (127.20.231.131:34207): tablet 28474855969c4ca6bb0ee277f1323df2 (table test-table [id=e8194ee6aaf7418697ca312cc76ea3fb]) successfully deleted
Jun 19 16:58:32 dist-test-slave-vnbj krb5kdc[23092](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888312, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 16:58:32 dist-test-slave-vnbj krb5kdc[23092](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888312, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:32.175264 23734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:32.162426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35404 (local address 127.20.231.190:45859)
0619 16:58:32.162587 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:32.162589 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:32.162599 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:58:32.162627 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:32.162628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:32.162645 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:32.162724 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:32.163359 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:32.163888 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:32.164653 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:32.164868 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:32.164945 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:32.172033 (+ 7088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:32.172035 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:32.172035 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:32.172045 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:32.173797 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:32.174208 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:32.174209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:32.174210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:32.174273 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:32.174500 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:32.174501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:32.174501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:32.174692 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:32.174778 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:32.175104 (+ 326us) server_negotiation.cc:299] Negotiation successful
0619 16:58:32.175191 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":83,"threads_started":1}
I20260619 16:58:32.177083 23137 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35404:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:58:32.177315 23137 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:58:32.179137 23414 tablet_service.cc:1467] Processing CreateTablet for tablet bd7e734f779e4b089bbf1ab265637402 (DEFAULT_TABLE table=test-table [id=7e674562431a4c7f92e6b9c00ba8f35a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:32.179137 23272 tablet_service.cc:1467] Processing CreateTablet for tablet bd7e734f779e4b089bbf1ab265637402 (DEFAULT_TABLE table=test-table [id=7e674562431a4c7f92e6b9c00ba8f35a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:32.179196 23554 tablet_service.cc:1467] Processing CreateTablet for tablet bd7e734f779e4b089bbf1ab265637402 (DEFAULT_TABLE table=test-table [id=7e674562431a4c7f92e6b9c00ba8f35a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:32.179298 23272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd7e734f779e4b089bbf1ab265637402. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:32.179332 23414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd7e734f779e4b089bbf1ab265637402. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:32.179353 23554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd7e734f779e4b089bbf1ab265637402. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:32.180547 23735 tablet_bootstrap.cc:492] T bd7e734f779e4b089bbf1ab265637402 P b17be09a87944e6f829f6198f4f21b8c: Bootstrap starting.
I20260619 16:58:32.180991 23737 tablet_bootstrap.cc:492] T bd7e734f779e4b089bbf1ab265637402 P af047fc88d734af0b2b6aad65d01bdb0: Bootstrap starting.
I20260619 16:58:32.181011 23735 tablet_bootstrap.cc:654] T bd7e734f779e4b089bbf1ab265637402 P b17be09a87944e6f829f6198f4f21b8c: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:32.181411 23737 tablet_bootstrap.cc:654] T bd7e734f779e4b089bbf1ab265637402 P af047fc88d734af0b2b6aad65d01bdb0: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:32.181717 23736 tablet_bootstrap.cc:492] T bd7e734f779e4b089bbf1ab265637402 P c84391626820478bbb21215c50fb899a: Bootstrap starting.
I20260619 16:58:32.181741 23735 tablet_bootstrap.cc:492] T bd7e734f779e4b089bbf1ab265637402 P b17be09a87944e6f829f6198f4f21b8c: No bootstrap required, opened a new log
I20260619 16:58:32.181795 23735 ts_tablet_manager.cc:1397] T bd7e734f779e4b089bbf1ab265637402 P b17be09a87944e6f829f6198f4f21b8c: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:32.181923 23735 raft_consensus.cc:348] T bd7e734f779e4b089bbf1ab265637402 P b17be09a87944e6f829f6198f4f21b8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:32.182024 23735 raft_consensus.cc:374] T bd7e734f779e4b089bbf1ab265637402 P b17be09a87944e6f829f6198f4f21b8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:32.182052 23735 raft_consensus.cc:729] T bd7e734f779e4b089bbf1ab265637402 P b17be09a87944e6f829f6198f4f21b8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b17be09a87944e6f829f6198f4f21b8c, State: Initialized, Role: FOLLOWER
I20260619 16:58:32.182057 23736 tablet_bootstrap.cc:654] T bd7e734f779e4b089bbf1ab265637402 P c84391626820478bbb21215c50fb899a: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:32.182062 23737 tablet_bootstrap.cc:492] T bd7e734f779e4b089bbf1ab265637402 P af047fc88d734af0b2b6aad65d01bdb0: No bootstrap required, opened a new log
I20260619 16:58:32.182106 23737 ts_tablet_manager.cc:1397] T bd7e734f779e4b089bbf1ab265637402 P af047fc88d734af0b2b6aad65d01bdb0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:32.182148 23735 consensus_queue.cc:260] T bd7e734f779e4b089bbf1ab265637402 P b17be09a87944e6f829f6198f4f21b8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:32.182308 23735 ts_tablet_manager.cc:1428] T bd7e734f779e4b089bbf1ab265637402 P b17be09a87944e6f829f6198f4f21b8c: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 16:58:32.182286 23737 raft_consensus.cc:348] T bd7e734f779e4b089bbf1ab265637402 P af047fc88d734af0b2b6aad65d01bdb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:32.182353 23737 raft_consensus.cc:374] T bd7e734f779e4b089bbf1ab265637402 P af047fc88d734af0b2b6aad65d01bdb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:32.182375 23737 raft_consensus.cc:729] T bd7e734f779e4b089bbf1ab265637402 P af047fc88d734af0b2b6aad65d01bdb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af047fc88d734af0b2b6aad65d01bdb0, State: Initialized, Role: FOLLOWER
I20260619 16:58:32.182421 23737 consensus_queue.cc:260] T bd7e734f779e4b089bbf1ab265637402 P af047fc88d734af0b2b6aad65d01bdb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:32.182524 23737 ts_tablet_manager.cc:1428] T bd7e734f779e4b089bbf1ab265637402 P af047fc88d734af0b2b6aad65d01bdb0: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:32.182621 23736 tablet_bootstrap.cc:492] T bd7e734f779e4b089bbf1ab265637402 P c84391626820478bbb21215c50fb899a: No bootstrap required, opened a new log
I20260619 16:58:32.182669 23736 ts_tablet_manager.cc:1397] T bd7e734f779e4b089bbf1ab265637402 P c84391626820478bbb21215c50fb899a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:32.182780 23736 raft_consensus.cc:348] T bd7e734f779e4b089bbf1ab265637402 P c84391626820478bbb21215c50fb899a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:32.182878 23736 raft_consensus.cc:374] T bd7e734f779e4b089bbf1ab265637402 P c84391626820478bbb21215c50fb899a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:32.182901 23736 raft_consensus.cc:729] T bd7e734f779e4b089bbf1ab265637402 P c84391626820478bbb21215c50fb899a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c84391626820478bbb21215c50fb899a, State: Initialized, Role: FOLLOWER
I20260619 16:58:32.182950 23736 consensus_queue.cc:260] T bd7e734f779e4b089bbf1ab265637402 P c84391626820478bbb21215c50fb899a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:32.183125 23736 ts_tablet_manager.cc:1428] T bd7e734f779e4b089bbf1ab265637402 P c84391626820478bbb21215c50fb899a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 16:58:32.272478 23715 raft_consensus.cc:482] T bd7e734f779e4b089bbf1ab265637402 P b17be09a87944e6f829f6198f4f21b8c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:58:32.272584 23715 raft_consensus.cc:504] T bd7e734f779e4b089bbf1ab265637402 P b17be09a87944e6f829f6198f4f21b8c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:32.272789 23715 leader_election.cc:288] T bd7e734f779e4b089bbf1ab265637402 P b17be09a87944e6f829f6198f4f21b8c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c84391626820478bbb21215c50fb899a (127.20.231.129:35139), af047fc88d734af0b2b6aad65d01bdb0 (127.20.231.130:34463)
I20260619 16:58:32.272969 23290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd7e734f779e4b089bbf1ab265637402" candidate_uuid: "b17be09a87944e6f829f6198f4f21b8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c84391626820478bbb21215c50fb899a" is_pre_election: true
I20260619 16:58:32.273038 23434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd7e734f779e4b089bbf1ab265637402" candidate_uuid: "b17be09a87944e6f829f6198f4f21b8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af047fc88d734af0b2b6aad65d01bdb0" is_pre_election: true
I20260619 16:58:32.273100 23290 raft_consensus.cc:2454] T bd7e734f779e4b089bbf1ab265637402 P c84391626820478bbb21215c50fb899a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b17be09a87944e6f829f6198f4f21b8c in term 0.
I20260619 16:58:32.273118 23434 raft_consensus.cc:2454] T bd7e734f779e4b089bbf1ab265637402 P af047fc88d734af0b2b6aad65d01bdb0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b17be09a87944e6f829f6198f4f21b8c in term 0.
I20260619 16:58:32.273346 23507 leader_election.cc:302] T bd7e734f779e4b089bbf1ab265637402 P b17be09a87944e6f829f6198f4f21b8c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af047fc88d734af0b2b6aad65d01bdb0, b17be09a87944e6f829f6198f4f21b8c; no voters:
I20260619 16:58:32.273484 23715 raft_consensus.cc:2781] T bd7e734f779e4b089bbf1ab265637402 P b17be09a87944e6f829f6198f4f21b8c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:58:32.273550 23715 raft_consensus.cc:482] T bd7e734f779e4b089bbf1ab265637402 P b17be09a87944e6f829f6198f4f21b8c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:58:32.273586 23715 raft_consensus.cc:3037] T bd7e734f779e4b089bbf1ab265637402 P b17be09a87944e6f829f6198f4f21b8c [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:32.274520 23715 raft_consensus.cc:504] T bd7e734f779e4b089bbf1ab265637402 P b17be09a87944e6f829f6198f4f21b8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:32.274673 23715 leader_election.cc:288] T bd7e734f779e4b089bbf1ab265637402 P b17be09a87944e6f829f6198f4f21b8c [CANDIDATE]: Term 1 election: Requested vote from peers c84391626820478bbb21215c50fb899a (127.20.231.129:35139), af047fc88d734af0b2b6aad65d01bdb0 (127.20.231.130:34463)
I20260619 16:58:32.274861 23290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd7e734f779e4b089bbf1ab265637402" candidate_uuid: "b17be09a87944e6f829f6198f4f21b8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c84391626820478bbb21215c50fb899a"
I20260619 16:58:32.274837 23434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd7e734f779e4b089bbf1ab265637402" candidate_uuid: "b17be09a87944e6f829f6198f4f21b8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af047fc88d734af0b2b6aad65d01bdb0"
I20260619 16:58:32.274930 23290 raft_consensus.cc:3037] T bd7e734f779e4b089bbf1ab265637402 P c84391626820478bbb21215c50fb899a [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:32.274971 23434 raft_consensus.cc:3037] T bd7e734f779e4b089bbf1ab265637402 P af047fc88d734af0b2b6aad65d01bdb0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:32.275655 23290 raft_consensus.cc:2454] T bd7e734f779e4b089bbf1ab265637402 P c84391626820478bbb21215c50fb899a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b17be09a87944e6f829f6198f4f21b8c in term 1.
I20260619 16:58:32.275681 23434 raft_consensus.cc:2454] T bd7e734f779e4b089bbf1ab265637402 P af047fc88d734af0b2b6aad65d01bdb0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b17be09a87944e6f829f6198f4f21b8c in term 1.
I20260619 16:58:32.275866 23507 leader_election.cc:302] T bd7e734f779e4b089bbf1ab265637402 P b17be09a87944e6f829f6198f4f21b8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af047fc88d734af0b2b6aad65d01bdb0, b17be09a87944e6f829f6198f4f21b8c; no voters:
I20260619 16:58:32.275967 23715 raft_consensus.cc:2781] T bd7e734f779e4b089bbf1ab265637402 P b17be09a87944e6f829f6198f4f21b8c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:32.276021 23715 raft_consensus.cc:686] T bd7e734f779e4b089bbf1ab265637402 P b17be09a87944e6f829f6198f4f21b8c [term 1 LEADER]: Becoming Leader. State: Replica: b17be09a87944e6f829f6198f4f21b8c, State: Running, Role: LEADER
I20260619 16:58:32.276084 23715 consensus_queue.cc:237] T bd7e734f779e4b089bbf1ab265637402 P b17be09a87944e6f829f6198f4f21b8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:32.276767 23136 catalog_manager.cc:5495] T bd7e734f779e4b089bbf1ab265637402 P b17be09a87944e6f829f6198f4f21b8c reported cstate change: term changed from 0 to 1, leader changed from <none> to b17be09a87944e6f829f6198f4f21b8c (127.20.231.131). New cstate: current_term: 1 leader_uuid: "b17be09a87944e6f829f6198f4f21b8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } health_report { overall_health: HEALTHY } } }
I20260619 16:58:32.303315 23744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:32.300757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37328 (local address 127.20.231.131:34207)
0619 16:58:32.300920 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:32.300922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:32.300939 (+ 17us) server_negotiation.cc:407] Connection header received
0619 16:58:32.300960 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:32.300961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:32.300971 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:32.301030 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:58:32.301158 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:32.301681 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:32.302541 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:32.302787 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:32.302894 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:32.303040 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:58:32.303071 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:58:32.303117 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:32.303230 (+ 113us) server_negotiation.cc:299] Negotiation successful
0619 16:58:32.303249 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":112,"threads_started":1}
I20260619 16:58:32.305696 23434 raft_consensus.cc:1261] T bd7e734f779e4b089bbf1ab265637402 P af047fc88d734af0b2b6aad65d01bdb0 [term 1 FOLLOWER]: Refusing update from remote peer b17be09a87944e6f829f6198f4f21b8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:32.305699 23290 raft_consensus.cc:1261] T bd7e734f779e4b089bbf1ab265637402 P c84391626820478bbb21215c50fb899a [term 1 FOLLOWER]: Refusing update from remote peer b17be09a87944e6f829f6198f4f21b8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:32.305948 23715 consensus_queue.cc:1035] T bd7e734f779e4b089bbf1ab265637402 P b17be09a87944e6f829f6198f4f21b8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:32.306072 23718 consensus_queue.cc:1035] T bd7e734f779e4b089bbf1ab265637402 P b17be09a87944e6f829f6198f4f21b8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:32.316398 23136 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:58:32.316505 23136 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:58:32.317214 23136 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d95249b6e2324671b047117583f5f601: Sending DeleteTablet for 3 replicas of tablet bd7e734f779e4b089bbf1ab265637402
I20260619 16:58:32.317512 23414 tablet_service.cc:1514] Processing DeleteTablet for tablet bd7e734f779e4b089bbf1ab265637402 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:32 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:56216
I20260619 16:58:32.317533 23554 tablet_service.cc:1514] Processing DeleteTablet for tablet bd7e734f779e4b089bbf1ab265637402 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:32 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:37310
I20260619 16:58:32.317545 23272 tablet_service.cc:1514] Processing DeleteTablet for tablet bd7e734f779e4b089bbf1ab265637402 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:32 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:47066
I20260619 16:58:32.317623 23726 tablet_replica.cc:330] T bd7e734f779e4b089bbf1ab265637402 P b17be09a87944e6f829f6198f4f21b8c: stopping tablet replica
I20260619 16:58:32.317623 23725 tablet_replica.cc:330] T bd7e734f779e4b089bbf1ab265637402 P af047fc88d734af0b2b6aad65d01bdb0: stopping tablet replica
I20260619 16:58:32.317677 23725 raft_consensus.cc:2229] T bd7e734f779e4b089bbf1ab265637402 P af047fc88d734af0b2b6aad65d01bdb0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:58:32.317730 23725 raft_consensus.cc:2258] T bd7e734f779e4b089bbf1ab265637402 P af047fc88d734af0b2b6aad65d01bdb0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:32.317680 23726 raft_consensus.cc:2229] T bd7e734f779e4b089bbf1ab265637402 P b17be09a87944e6f829f6198f4f21b8c [term 1 LEADER]: Raft consensus shutting down.
I20260619 16:58:32.317770 23724 tablet_replica.cc:330] T bd7e734f779e4b089bbf1ab265637402 P c84391626820478bbb21215c50fb899a: stopping tablet replica
I20260619 16:58:32.317800 23726 raft_consensus.cc:2258] T bd7e734f779e4b089bbf1ab265637402 P b17be09a87944e6f829f6198f4f21b8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:32.317845 23724 raft_consensus.cc:2229] T bd7e734f779e4b089bbf1ab265637402 P c84391626820478bbb21215c50fb899a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:58:32.317904 23724 raft_consensus.cc:2258] T bd7e734f779e4b089bbf1ab265637402 P c84391626820478bbb21215c50fb899a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:32.317991 23136 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35404:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 16:58:32.318104 23725 ts_tablet_manager.cc:1905] T bd7e734f779e4b089bbf1ab265637402 P af047fc88d734af0b2b6aad65d01bdb0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 16:58:32.318141 23136 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:58:32.318156 23726 ts_tablet_manager.cc:1905] T bd7e734f779e4b089bbf1ab265637402 P b17be09a87944e6f829f6198f4f21b8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:32.318383 23724 ts_tablet_manager.cc:1905] T bd7e734f779e4b089bbf1ab265637402 P c84391626820478bbb21215c50fb899a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:32.319342 23272 tablet_service.cc:1467] Processing CreateTablet for tablet cf7233b77d314199802f3cf89b37d5d7 (DEFAULT_TABLE table=test-table [id=0aff1ee80a6046ceaa3d9bbf245bc51f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:32.319461 23272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf7233b77d314199802f3cf89b37d5d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:32.319943 23725 ts_tablet_manager.cc:1918] T bd7e734f779e4b089bbf1ab265637402 P af047fc88d734af0b2b6aad65d01bdb0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 16:58:32.319996 23725 log.cc:1196] T bd7e734f779e4b089bbf1ab265637402 P af047fc88d734af0b2b6aad65d01bdb0: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-1/wal/wals/bd7e734f779e4b089bbf1ab265637402
I20260619 16:58:32.320129 23724 ts_tablet_manager.cc:1918] T bd7e734f779e4b089bbf1ab265637402 P c84391626820478bbb21215c50fb899a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 16:58:32.320189 23724 log.cc:1196] T bd7e734f779e4b089bbf1ab265637402 P c84391626820478bbb21215c50fb899a: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-0/wal/wals/bd7e734f779e4b089bbf1ab265637402
I20260619 16:58:32.320255 23725 ts_tablet_manager.cc:1939] T bd7e734f779e4b089bbf1ab265637402 P af047fc88d734af0b2b6aad65d01bdb0: Deleting consensus metadata
I20260619 16:58:32.320290 23726 ts_tablet_manager.cc:1918] T bd7e734f779e4b089bbf1ab265637402 P b17be09a87944e6f829f6198f4f21b8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 16:58:32.320346 23726 log.cc:1196] T bd7e734f779e4b089bbf1ab265637402 P b17be09a87944e6f829f6198f4f21b8c: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-2/wal/wals/bd7e734f779e4b089bbf1ab265637402
I20260619 16:58:32.320456 23724 ts_tablet_manager.cc:1939] T bd7e734f779e4b089bbf1ab265637402 P c84391626820478bbb21215c50fb899a: Deleting consensus metadata
I20260619 16:58:32.320621 23726 ts_tablet_manager.cc:1939] T bd7e734f779e4b089bbf1ab265637402 P b17be09a87944e6f829f6198f4f21b8c: Deleting consensus metadata
I20260619 16:58:32.320824 23736 tablet_bootstrap.cc:492] T cf7233b77d314199802f3cf89b37d5d7 P c84391626820478bbb21215c50fb899a: Bootstrap starting.
I20260619 16:58:32.320819 23121 catalog_manager.cc:4841] TS c84391626820478bbb21215c50fb899a (127.20.231.129:35139): tablet bd7e734f779e4b089bbf1ab265637402 (table test-table [id=7e674562431a4c7f92e6b9c00ba8f35a]) successfully deleted
I20260619 16:58:32.321225 23736 tablet_bootstrap.cc:654] T cf7233b77d314199802f3cf89b37d5d7 P c84391626820478bbb21215c50fb899a: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:32.321583 23123 catalog_manager.cc:4841] TS af047fc88d734af0b2b6aad65d01bdb0 (127.20.231.130:34463): tablet bd7e734f779e4b089bbf1ab265637402 (table test-table [id=7e674562431a4c7f92e6b9c00ba8f35a]) successfully deleted
I20260619 16:58:32.321801 23414 tablet_service.cc:1467] Processing CreateTablet for tablet cf7233b77d314199802f3cf89b37d5d7 (DEFAULT_TABLE table=test-table [id=0aff1ee80a6046ceaa3d9bbf245bc51f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:32.321869 23122 catalog_manager.cc:4841] TS b17be09a87944e6f829f6198f4f21b8c (127.20.231.131:34207): tablet bd7e734f779e4b089bbf1ab265637402 (table test-table [id=7e674562431a4c7f92e6b9c00ba8f35a]) successfully deleted
I20260619 16:58:32.321923 23414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf7233b77d314199802f3cf89b37d5d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:32.321978 23554 tablet_service.cc:1467] Processing CreateTablet for tablet cf7233b77d314199802f3cf89b37d5d7 (DEFAULT_TABLE table=test-table [id=0aff1ee80a6046ceaa3d9bbf245bc51f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:32.322085 23554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cf7233b77d314199802f3cf89b37d5d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:32.323102 23737 tablet_bootstrap.cc:492] T cf7233b77d314199802f3cf89b37d5d7 P af047fc88d734af0b2b6aad65d01bdb0: Bootstrap starting.
I20260619 16:58:32.323292 23735 tablet_bootstrap.cc:492] T cf7233b77d314199802f3cf89b37d5d7 P b17be09a87944e6f829f6198f4f21b8c: Bootstrap starting.
I20260619 16:58:32.323472 23737 tablet_bootstrap.cc:654] T cf7233b77d314199802f3cf89b37d5d7 P af047fc88d734af0b2b6aad65d01bdb0: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:32.323617 23735 tablet_bootstrap.cc:654] T cf7233b77d314199802f3cf89b37d5d7 P b17be09a87944e6f829f6198f4f21b8c: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:32.324059 23736 tablet_bootstrap.cc:492] T cf7233b77d314199802f3cf89b37d5d7 P c84391626820478bbb21215c50fb899a: No bootstrap required, opened a new log
I20260619 16:58:32.324095 23737 tablet_bootstrap.cc:492] T cf7233b77d314199802f3cf89b37d5d7 P af047fc88d734af0b2b6aad65d01bdb0: No bootstrap required, opened a new log
I20260619 16:58:32.324126 23736 ts_tablet_manager.cc:1397] T cf7233b77d314199802f3cf89b37d5d7 P c84391626820478bbb21215c50fb899a: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 16:58:32.324141 23737 ts_tablet_manager.cc:1397] T cf7233b77d314199802f3cf89b37d5d7 P af047fc88d734af0b2b6aad65d01bdb0: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:32.324254 23737 raft_consensus.cc:348] T cf7233b77d314199802f3cf89b37d5d7 P af047fc88d734af0b2b6aad65d01bdb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:32.324254 23736 raft_consensus.cc:348] T cf7233b77d314199802f3cf89b37d5d7 P c84391626820478bbb21215c50fb899a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:32.324333 23737 raft_consensus.cc:374] T cf7233b77d314199802f3cf89b37d5d7 P af047fc88d734af0b2b6aad65d01bdb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:32.324333 23736 raft_consensus.cc:374] T cf7233b77d314199802f3cf89b37d5d7 P c84391626820478bbb21215c50fb899a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:32.324358 23736 raft_consensus.cc:729] T cf7233b77d314199802f3cf89b37d5d7 P c84391626820478bbb21215c50fb899a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c84391626820478bbb21215c50fb899a, State: Initialized, Role: FOLLOWER
I20260619 16:58:32.324358 23737 raft_consensus.cc:729] T cf7233b77d314199802f3cf89b37d5d7 P af047fc88d734af0b2b6aad65d01bdb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af047fc88d734af0b2b6aad65d01bdb0, State: Initialized, Role: FOLLOWER
I20260619 16:58:32.324405 23736 consensus_queue.cc:260] T cf7233b77d314199802f3cf89b37d5d7 P c84391626820478bbb21215c50fb899a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:32.324407 23737 consensus_queue.cc:260] T cf7233b77d314199802f3cf89b37d5d7 P af047fc88d734af0b2b6aad65d01bdb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:32.324507 23737 ts_tablet_manager.cc:1428] T cf7233b77d314199802f3cf89b37d5d7 P af047fc88d734af0b2b6aad65d01bdb0: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 16:58:32.324507 23736 ts_tablet_manager.cc:1428] T cf7233b77d314199802f3cf89b37d5d7 P c84391626820478bbb21215c50fb899a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 16:58:32.324991 23735 tablet_bootstrap.cc:492] T cf7233b77d314199802f3cf89b37d5d7 P b17be09a87944e6f829f6198f4f21b8c: No bootstrap required, opened a new log
I20260619 16:58:32.325034 23735 ts_tablet_manager.cc:1397] T cf7233b77d314199802f3cf89b37d5d7 P b17be09a87944e6f829f6198f4f21b8c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:58:32.325173 23735 raft_consensus.cc:348] T cf7233b77d314199802f3cf89b37d5d7 P b17be09a87944e6f829f6198f4f21b8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:32.325243 23735 raft_consensus.cc:374] T cf7233b77d314199802f3cf89b37d5d7 P b17be09a87944e6f829f6198f4f21b8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:32.325265 23735 raft_consensus.cc:729] T cf7233b77d314199802f3cf89b37d5d7 P b17be09a87944e6f829f6198f4f21b8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b17be09a87944e6f829f6198f4f21b8c, State: Initialized, Role: FOLLOWER
I20260619 16:58:32.325326 23735 consensus_queue.cc:260] T cf7233b77d314199802f3cf89b37d5d7 P b17be09a87944e6f829f6198f4f21b8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:32.325424 23735 ts_tablet_manager.cc:1428] T cf7233b77d314199802f3cf89b37d5d7 P b17be09a87944e6f829f6198f4f21b8c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:32.366354 23715 raft_consensus.cc:482] T cf7233b77d314199802f3cf89b37d5d7 P b17be09a87944e6f829f6198f4f21b8c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:58:32.366457 23715 raft_consensus.cc:504] T cf7233b77d314199802f3cf89b37d5d7 P b17be09a87944e6f829f6198f4f21b8c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:32.366626 23715 leader_election.cc:288] T cf7233b77d314199802f3cf89b37d5d7 P b17be09a87944e6f829f6198f4f21b8c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c84391626820478bbb21215c50fb899a (127.20.231.129:35139), af047fc88d734af0b2b6aad65d01bdb0 (127.20.231.130:34463)
I20260619 16:58:32.366884 23290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf7233b77d314199802f3cf89b37d5d7" candidate_uuid: "b17be09a87944e6f829f6198f4f21b8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c84391626820478bbb21215c50fb899a" is_pre_election: true
I20260619 16:58:32.366990 23290 raft_consensus.cc:2454] T cf7233b77d314199802f3cf89b37d5d7 P c84391626820478bbb21215c50fb899a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b17be09a87944e6f829f6198f4f21b8c in term 0.
I20260619 16:58:32.366940 23434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf7233b77d314199802f3cf89b37d5d7" candidate_uuid: "b17be09a87944e6f829f6198f4f21b8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af047fc88d734af0b2b6aad65d01bdb0" is_pre_election: true
I20260619 16:58:32.367054 23434 raft_consensus.cc:2454] T cf7233b77d314199802f3cf89b37d5d7 P af047fc88d734af0b2b6aad65d01bdb0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b17be09a87944e6f829f6198f4f21b8c in term 0.
I20260619 16:58:32.367178 23505 leader_election.cc:302] T cf7233b77d314199802f3cf89b37d5d7 P b17be09a87944e6f829f6198f4f21b8c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b17be09a87944e6f829f6198f4f21b8c, c84391626820478bbb21215c50fb899a; no voters:
I20260619 16:58:32.367322 23715 raft_consensus.cc:2781] T cf7233b77d314199802f3cf89b37d5d7 P b17be09a87944e6f829f6198f4f21b8c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:58:32.367396 23715 raft_consensus.cc:482] T cf7233b77d314199802f3cf89b37d5d7 P b17be09a87944e6f829f6198f4f21b8c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:58:32.367432 23715 raft_consensus.cc:3037] T cf7233b77d314199802f3cf89b37d5d7 P b17be09a87944e6f829f6198f4f21b8c [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:32.368125 23715 raft_consensus.cc:504] T cf7233b77d314199802f3cf89b37d5d7 P b17be09a87944e6f829f6198f4f21b8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:32.368287 23715 leader_election.cc:288] T cf7233b77d314199802f3cf89b37d5d7 P b17be09a87944e6f829f6198f4f21b8c [CANDIDATE]: Term 1 election: Requested vote from peers c84391626820478bbb21215c50fb899a (127.20.231.129:35139), af047fc88d734af0b2b6aad65d01bdb0 (127.20.231.130:34463)
I20260619 16:58:32.368425 23290 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf7233b77d314199802f3cf89b37d5d7" candidate_uuid: "b17be09a87944e6f829f6198f4f21b8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c84391626820478bbb21215c50fb899a"
I20260619 16:58:32.368520 23290 raft_consensus.cc:3037] T cf7233b77d314199802f3cf89b37d5d7 P c84391626820478bbb21215c50fb899a [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:32.368486 23434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cf7233b77d314199802f3cf89b37d5d7" candidate_uuid: "b17be09a87944e6f829f6198f4f21b8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af047fc88d734af0b2b6aad65d01bdb0"
I20260619 16:58:32.368594 23434 raft_consensus.cc:3037] T cf7233b77d314199802f3cf89b37d5d7 P af047fc88d734af0b2b6aad65d01bdb0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:32.369168 23290 raft_consensus.cc:2454] T cf7233b77d314199802f3cf89b37d5d7 P c84391626820478bbb21215c50fb899a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b17be09a87944e6f829f6198f4f21b8c in term 1.
I20260619 16:58:32.369206 23434 raft_consensus.cc:2454] T cf7233b77d314199802f3cf89b37d5d7 P af047fc88d734af0b2b6aad65d01bdb0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b17be09a87944e6f829f6198f4f21b8c in term 1.
I20260619 16:58:32.369405 23507 leader_election.cc:302] T cf7233b77d314199802f3cf89b37d5d7 P b17be09a87944e6f829f6198f4f21b8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: af047fc88d734af0b2b6aad65d01bdb0, b17be09a87944e6f829f6198f4f21b8c; no voters:
I20260619 16:58:32.369518 23715 raft_consensus.cc:2781] T cf7233b77d314199802f3cf89b37d5d7 P b17be09a87944e6f829f6198f4f21b8c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:32.369606 23715 raft_consensus.cc:686] T cf7233b77d314199802f3cf89b37d5d7 P b17be09a87944e6f829f6198f4f21b8c [term 1 LEADER]: Becoming Leader. State: Replica: b17be09a87944e6f829f6198f4f21b8c, State: Running, Role: LEADER
I20260619 16:58:32.369669 23715 consensus_queue.cc:237] T cf7233b77d314199802f3cf89b37d5d7 P b17be09a87944e6f829f6198f4f21b8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } }
I20260619 16:58:32.370337 23137 catalog_manager.cc:5495] T cf7233b77d314199802f3cf89b37d5d7 P b17be09a87944e6f829f6198f4f21b8c reported cstate change: term changed from 0 to 1, leader changed from <none> to b17be09a87944e6f829f6198f4f21b8c (127.20.231.131). New cstate: current_term: 1 leader_uuid: "b17be09a87944e6f829f6198f4f21b8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b17be09a87944e6f829f6198f4f21b8c" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 34207 } health_report { overall_health: HEALTHY } } }
I20260619 16:58:32.376368 23434 raft_consensus.cc:1261] T cf7233b77d314199802f3cf89b37d5d7 P af047fc88d734af0b2b6aad65d01bdb0 [term 1 FOLLOWER]: Refusing update from remote peer b17be09a87944e6f829f6198f4f21b8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:32.376422 23290 raft_consensus.cc:1261] T cf7233b77d314199802f3cf89b37d5d7 P c84391626820478bbb21215c50fb899a [term 1 FOLLOWER]: Refusing update from remote peer b17be09a87944e6f829f6198f4f21b8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:32.376627 23715 consensus_queue.cc:1035] T cf7233b77d314199802f3cf89b37d5d7 P b17be09a87944e6f829f6198f4f21b8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "af047fc88d734af0b2b6aad65d01bdb0" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:32.376724 23718 consensus_queue.cc:1035] T cf7233b77d314199802f3cf89b37d5d7 P b17be09a87944e6f829f6198f4f21b8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c84391626820478bbb21215c50fb899a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 16:58:32.381127 23707 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 16:58:33.898275 23136 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:58:33.898375 23136 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:58:33.899377 23136 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d95249b6e2324671b047117583f5f601: Sending DeleteTablet for 3 replicas of tablet cf7233b77d314199802f3cf89b37d5d7
I20260619 16:58:33.899645 23414 tablet_service.cc:1514] Processing DeleteTablet for tablet cf7233b77d314199802f3cf89b37d5d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:33 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:56216
I20260619 16:58:33.899716 23554 tablet_service.cc:1514] Processing DeleteTablet for tablet cf7233b77d314199802f3cf89b37d5d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:33 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:37310
I20260619 16:58:33.899819 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 23209
I20260619 16:58:33.899848 23272 tablet_service.cc:1514] Processing DeleteTablet for tablet cf7233b77d314199802f3cf89b37d5d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:33 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:47066
I20260619 16:58:33.900974 23776 tablet_replica.cc:330] T cf7233b77d314199802f3cf89b37d5d7 P af047fc88d734af0b2b6aad65d01bdb0: stopping tablet replica
I20260619 16:58:33.901036 23777 tablet_replica.cc:330] T cf7233b77d314199802f3cf89b37d5d7 P b17be09a87944e6f829f6198f4f21b8c: stopping tablet replica
I20260619 16:58:33.901047 23776 raft_consensus.cc:2229] T cf7233b77d314199802f3cf89b37d5d7 P af047fc88d734af0b2b6aad65d01bdb0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:58:33.901086 23777 raft_consensus.cc:2229] T cf7233b77d314199802f3cf89b37d5d7 P b17be09a87944e6f829f6198f4f21b8c [term 1 LEADER]: Raft consensus shutting down.
I20260619 16:58:33.901095 23776 raft_consensus.cc:2258] T cf7233b77d314199802f3cf89b37d5d7 P af047fc88d734af0b2b6aad65d01bdb0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:33.901196 23777 raft_consensus.cc:2258] T cf7233b77d314199802f3cf89b37d5d7 P b17be09a87944e6f829f6198f4f21b8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:33.901386 23776 ts_tablet_manager.cc:1905] T cf7233b77d314199802f3cf89b37d5d7 P af047fc88d734af0b2b6aad65d01bdb0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:33.901432 23777 ts_tablet_manager.cc:1905] T cf7233b77d314199802f3cf89b37d5d7 P b17be09a87944e6f829f6198f4f21b8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:33.903026 23776 ts_tablet_manager.cc:1918] T cf7233b77d314199802f3cf89b37d5d7 P af047fc88d734af0b2b6aad65d01bdb0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 16:58:33.903036 23777 ts_tablet_manager.cc:1918] T cf7233b77d314199802f3cf89b37d5d7 P b17be09a87944e6f829f6198f4f21b8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 16:58:33.903093 23777 log.cc:1196] T cf7233b77d314199802f3cf89b37d5d7 P b17be09a87944e6f829f6198f4f21b8c: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-2/wal/wals/cf7233b77d314199802f3cf89b37d5d7
I20260619 16:58:33.903093 23776 log.cc:1196] T cf7233b77d314199802f3cf89b37d5d7 P af047fc88d734af0b2b6aad65d01bdb0: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781888301345144-21406-0/minicluster-data/ts-1/wal/wals/cf7233b77d314199802f3cf89b37d5d7
I20260619 16:58:33.903337 23777 ts_tablet_manager.cc:1939] T cf7233b77d314199802f3cf89b37d5d7 P b17be09a87944e6f829f6198f4f21b8c: Deleting consensus metadata
I20260619 16:58:33.903334 23776 ts_tablet_manager.cc:1939] T cf7233b77d314199802f3cf89b37d5d7 P af047fc88d734af0b2b6aad65d01bdb0: Deleting consensus metadata
I20260619 16:58:33.903749 23123 catalog_manager.cc:4841] TS af047fc88d734af0b2b6aad65d01bdb0 (127.20.231.130:34463): tablet cf7233b77d314199802f3cf89b37d5d7 (table test-table [id=0aff1ee80a6046ceaa3d9bbf245bc51f]) successfully deleted
I20260619 16:58:33.903767 23122 catalog_manager.cc:4841] TS b17be09a87944e6f829f6198f4f21b8c (127.20.231.131:34207): tablet cf7233b77d314199802f3cf89b37d5d7 (table test-table [id=0aff1ee80a6046ceaa3d9bbf245bc51f]) successfully deleted
W20260619 16:58:33.907051 23121 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.231.129:35139 (error 108)
I20260619 16:58:33.907232 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 23349
I20260619 16:58:33.908160 23779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:33.907917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:35139 (local address 127.0.0.1:47090)
0619 16:58:33.908069 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:33.908119 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.129:35139: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":70,"threads_started":1}
W20260619 16:58:33.908309 23121 catalog_manager.cc:4584] TS c84391626820478bbb21215c50fb899a (127.20.231.129:35139): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cf7233b77d314199802f3cf89b37d5d7: Network error: Client connection negotiation failed: client connection to 127.20.231.129:35139: connect: Connection refused (error 111)
I20260619 16:58:33.914394 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 23489
I20260619 16:58:33.919740 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 23105
2026-06-19T16:58:33Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5591 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:58:33 dist-test-slave-vnbj krb5kdc[23782](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:58:33 dist-test-slave-vnbj krb5kdc[23782](info): set up 2 sockets
Jun 19 16:58:33 dist-test-slave-vnbj krb5kdc[23782](info): commencing operation
krb5kdc: starting...
W20260619 16:58:34.991397 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 16:58:35 dist-test-slave-vnbj krb5kdc[23782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:58:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:58:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.231.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
I20260619 16:58:35.119354 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:44201
--webserver_interface=127.20.231.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:39445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:44201
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:35.203344 23795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:35.203572 23795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:35.203596 23795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:35.204909 23795 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:35.204962 23795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:35.204977 23795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:35.204990 23795 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:35.205003 23795 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:58:35.206686 23795 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:39445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:44201
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:44201
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:35.206995 23795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:35.207275 23795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 16:58:35.213020 23795 server_base.cc:1034] running on GCE node
W20260619 16:58:35.212961 23800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:35.212981 23803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:35.212975 23801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:35.213584 23795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:35.213932 23795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:35.215106 23795 hybrid_clock.cc:648] HybridClock initialized: now 1781888315215086 us; error 47 us; skew 500 ppm
Jun 19 16:58:35 dist-test-slave-vnbj krb5kdc[23782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888315, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:35.217983 23795 init.cc:377] Logged in from keytab as kudu/127.20.231.190@KRBTEST.COM (short username kudu)
I20260619 16:58:35.218443 23795 webserver.cc:460] Webserver started at http://127.20.231.190:37749/ using document root <none> and password file <none>
I20260619 16:58:35.218676 23795 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:35.218721 23795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:35.218906 23795 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:35.220054 23795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "03ac67173840401bb2afd4adc471f9ac"
format_stamp: "Formatted at 2026-06-19 16:58:35 on dist-test-slave-vnbj"
server_key: "4a1975319afffdfc374a6468ba3aed8d"
server_key_iv: "c3a425fda6128cad81550c076d317975"
server_key_version: "encryptionkey@0"
I20260619 16:58:35.220461 23795 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "03ac67173840401bb2afd4adc471f9ac"
format_stamp: "Formatted at 2026-06-19 16:58:35 on dist-test-slave-vnbj"
server_key: "4a1975319afffdfc374a6468ba3aed8d"
server_key_iv: "c3a425fda6128cad81550c076d317975"
server_key_version: "encryptionkey@0"
I20260619 16:58:35.221964 23795 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:58:35.222806 23810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:35.223073 23795 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:58:35.223158 23795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "03ac67173840401bb2afd4adc471f9ac"
format_stamp: "Formatted at 2026-06-19 16:58:35 on dist-test-slave-vnbj"
server_key: "4a1975319afffdfc374a6468ba3aed8d"
server_key_iv: "c3a425fda6128cad81550c076d317975"
server_key_version: "encryptionkey@0"
I20260619 16:58:35.223250 23795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:35.248692 23795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:35.251252 23795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:35.251426 23795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:35.255432 23795 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:44201
I20260619 16:58:35.255501 23862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:44201 every 8 connection(s)
I20260619 16:58:35.255842 23795 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 16:58:35.256850 23863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:35.259238 23863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03ac67173840401bb2afd4adc471f9ac: Bootstrap starting.
I20260619 16:58:35.259851 23863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03ac67173840401bb2afd4adc471f9ac: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:35.260090 23863 log.cc:824] T 00000000000000000000000000000000 P 03ac67173840401bb2afd4adc471f9ac: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:35.260865 23863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03ac67173840401bb2afd4adc471f9ac: No bootstrap required, opened a new log
I20260619 16:58:35.262733 23863 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03ac67173840401bb2afd4adc471f9ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ac67173840401bb2afd4adc471f9ac" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 44201 } }
I20260619 16:58:35.262921 23863 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03ac67173840401bb2afd4adc471f9ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:35.262951 23863 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03ac67173840401bb2afd4adc471f9ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03ac67173840401bb2afd4adc471f9ac, State: Initialized, Role: FOLLOWER
I20260619 16:58:35.263150 23863 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03ac67173840401bb2afd4adc471f9ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ac67173840401bb2afd4adc471f9ac" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 44201 } }
I20260619 16:58:35.263253 23863 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03ac67173840401bb2afd4adc471f9ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:58:35.263283 23863 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03ac67173840401bb2afd4adc471f9ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:58:35.263324 23863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03ac67173840401bb2afd4adc471f9ac [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:35.263934 23863 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03ac67173840401bb2afd4adc471f9ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ac67173840401bb2afd4adc471f9ac" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 44201 } }
I20260619 16:58:35.264034 23863 leader_election.cc:302] T 00000000000000000000000000000000 P 03ac67173840401bb2afd4adc471f9ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03ac67173840401bb2afd4adc471f9ac; no voters:
I20260619 16:58:35.264216 23863 leader_election.cc:288] T 00000000000000000000000000000000 P 03ac67173840401bb2afd4adc471f9ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:58:35.264338 23866 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03ac67173840401bb2afd4adc471f9ac [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:35.264485 23866 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03ac67173840401bb2afd4adc471f9ac [term 1 LEADER]: Becoming Leader. State: Replica: 03ac67173840401bb2afd4adc471f9ac, State: Running, Role: LEADER
I20260619 16:58:35.264542 23863 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03ac67173840401bb2afd4adc471f9ac [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:35.264572 23866 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03ac67173840401bb2afd4adc471f9ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ac67173840401bb2afd4adc471f9ac" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 44201 } }
I20260619 16:58:35.264887 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 23795
I20260619 16:58:35.265002 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 16:58:35.265251 21406 external_mini_cluster.cc:1428] Setting key 60335f1bb0d5d7d61d604e429010c7a7
I20260619 16:58:35.266060 23867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03ac67173840401bb2afd4adc471f9ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03ac67173840401bb2afd4adc471f9ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ac67173840401bb2afd4adc471f9ac" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 44201 } } }
I20260619 16:58:35.266196 23867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03ac67173840401bb2afd4adc471f9ac [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:35.266425 23868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03ac67173840401bb2afd4adc471f9ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03ac67173840401bb2afd4adc471f9ac. Latest consensus state: current_term: 1 leader_uuid: "03ac67173840401bb2afd4adc471f9ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03ac67173840401bb2afd4adc471f9ac" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 44201 } } }
I20260619 16:58:35.266571 23868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03ac67173840401bb2afd4adc471f9ac [sys.catalog]: This master's current role is: LEADER
W20260619 16:58:35.266923 23884 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 03ac67173840401bb2afd4adc471f9ac: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 16:58:35.266986 23884 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 16:58:35.267069 23883 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:58:35.267236 23883 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:58:35.269122 23883 catalog_manager.cc:1347] Generated new cluster ID: d0dd03928c7a494ca2ccda557c74be33
I20260619 16:58:35.269189 23883 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 16:58:35 dist-test-slave-vnbj krb5kdc[23782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:35.278246 23883 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:58:35.279062 23883 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:58:35.279356 23878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:35.265814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35948 (local address 127.20.231.190:44201)
0619 16:58:35.265983 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:35.265996 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:35.266826 (+ 830us) server_negotiation.cc:407] Connection header received
0619 16:58:35.267093 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:35.267101 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:35.267163 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:35.267323 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:35.268271 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:35.269808 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:35.270990 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:35.271712 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:35.271829 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:35.274987 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:35.274993 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:35.274993 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:35.275007 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:35.277217 (+ 2210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:35.277781 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:35.277785 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:35.277786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:35.277926 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:35.278245 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:35.278246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:35.278246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:35.278602 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:35.278788 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:35.278927 (+ 139us) server_negotiation.cc:299] Negotiation successful
0619 16:58:35.279100 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":90,"threads_started":1}
I20260619 16:58:35.286469 23883 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03ac67173840401bb2afd4adc471f9ac: Generated new TSK 0
I20260619 16:58:35.286716 23883 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.231.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab.
Entry for principal kudu/127.20.231.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab.
I20260619 16:58:35.321619 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.129:0
--local_ip_for_outbound_sockets=127.20.231.129
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:44201
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:39445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:35.404557 23889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:35.404770 23889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:35.404794 23889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:35.406014 23889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:35.406104 23889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:35.406153 23889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.129
I20260619 16:58:35.407701 23889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:39445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:44201
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.129
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:35.407966 23889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:35.408239 23889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:35.413820 23894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:35.413873 23897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:35.413874 23895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:35.413900 23889 server_base.cc:1034] running on GCE node
I20260619 16:58:35.414256 23889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:35.414508 23889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:35.415671 23889 hybrid_clock.cc:648] HybridClock initialized: now 1781888315415667 us; error 40 us; skew 500 ppm
Jun 19 16:58:35 dist-test-slave-vnbj krb5kdc[23782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888315, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:35.418315 23889 init.cc:377] Logged in from keytab as kudu/127.20.231.129@KRBTEST.COM (short username kudu)
I20260619 16:58:35.418900 23889 webserver.cc:460] Webserver started at http://127.20.231.129:41697/ using document root <none> and password file <none>
I20260619 16:58:35.419160 23889 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:35.419231 23889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:35.419389 23889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:35.420521 23889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-0/data/instance:
uuid: "1ed85068c46f4b4f8acfea4004c6078b"
format_stamp: "Formatted at 2026-06-19 16:58:35 on dist-test-slave-vnbj"
server_key: "ffa27476d80f30359173994e5333b4ec"
server_key_iv: "dd1659c0b7be81545d7cb119fde1bef7"
server_key_version: "encryptionkey@0"
I20260619 16:58:35.420895 23889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance:
uuid: "1ed85068c46f4b4f8acfea4004c6078b"
format_stamp: "Formatted at 2026-06-19 16:58:35 on dist-test-slave-vnbj"
server_key: "ffa27476d80f30359173994e5333b4ec"
server_key_iv: "dd1659c0b7be81545d7cb119fde1bef7"
server_key_version: "encryptionkey@0"
I20260619 16:58:35.422242 23889 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:58:35.423179 23904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:35.423362 23889 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:35.423466 23889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-0/wal
uuid: "1ed85068c46f4b4f8acfea4004c6078b"
format_stamp: "Formatted at 2026-06-19 16:58:35 on dist-test-slave-vnbj"
server_key: "ffa27476d80f30359173994e5333b4ec"
server_key_iv: "dd1659c0b7be81545d7cb119fde1bef7"
server_key_version: "encryptionkey@0"
I20260619 16:58:35.423540 23889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:35.432080 23889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:35.434206 23889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:35.434371 23889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:35.434607 23889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:58:35.434978 23889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:35.435031 23889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:35.435065 23889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:35.435092 23889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:35.441268 23889 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.129:43441
I20260619 16:58:35.441370 24017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.129:43441 every 8 connection(s)
I20260619 16:58:35.441680 23889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
I20260619 16:58:35.447000 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 23889
I20260619 16:58:35.447088 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance
I20260619 16:58:35.447296 21406 external_mini_cluster.cc:1428] Setting key d5885e5cf2251a1fbb59b36479199ec6
Jun 19 16:58:35 dist-test-slave-vnbj krb5kdc[23782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888315, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:35.452423 24020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:35.442410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:44201 (local address 127.20.231.129:44949)
0619 16:58:35.442565 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:35.442605 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:35.443169 (+ 564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:35.443437 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:35.443441 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:35.443995 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:35.444185 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:35.444190 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:35.444937 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:35.444938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:35.445738 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:35.445740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:35.446215 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:35.446216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:35.446337 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:35.446995 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:35.447003 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:35.449006 (+ 2003us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:35.451004 (+ 1998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:35.451010 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:35.451012 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:35.451231 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:35.451454 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:35.451455 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:35.451455 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:35.451552 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:35.451857 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:35.451861 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:35.451976 (+ 115us) client_negotiation.cc:769] Sending connection context
0619 16:58:35.452113 (+ 137us) client_negotiation.cc:240] Negotiation successful
0619 16:58:35.452256 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":88,"threads_started":1}
I20260619 16:58:35.452435 23878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:35.442461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:44949 (local address 127.20.231.190:44201)
0619 16:58:35.442506 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:35.442508 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:35.442735 (+ 227us) server_negotiation.cc:407] Connection header received
0619 16:58:35.443291 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:35.443293 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:35.443305 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:35.443383 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:35.444309 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:35.444846 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:35.445846 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:35.446108 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:35.446200 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:35.449106 (+ 2906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:35.449108 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:35.449109 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:35.449126 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:35.450926 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:35.451314 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:35.451315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:35.451316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:35.451385 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:35.451620 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:35.451621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:35.451621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:35.451785 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:35.451865 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:35.452191 (+ 326us) server_negotiation.cc:299] Negotiation successful
0619 16:58:35.452298 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 16:58:35.453087 24018 heartbeater.cc:347] Connected to a master server at 127.20.231.190:44201
I20260619 16:58:35.453187 24018 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:35.453383 24018 heartbeater.cc:511] Master 127.20.231.190:44201 requested a full tablet report, sending...
I20260619 16:58:35.453922 23827 ts_manager.cc:194] Registered new tserver with Master: 1ed85068c46f4b4f8acfea4004c6078b (127.20.231.129:43441)
I20260619 16:58:35.454581 23827 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.129@KRBTEST.COM'} at 127.20.231.129:44949
WARNING: no policy specified for kudu/127.20.231.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab.
Entry for principal kudu/127.20.231.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab.
I20260619 16:58:35.473996 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.130:0
--local_ip_for_outbound_sockets=127.20.231.130
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:44201
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:39445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:35.580524 24023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:35.580788 24023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:35.580832 24023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:35.582044 24023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:35.582121 24023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:35.582163 24023 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.130
I20260619 16:58:35.583832 24023 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:39445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:44201
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24023
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.130
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:35.584131 24023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:35.584368 24023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:35.589712 24028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:35.589735 24031 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:35.589673 24029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:35.590063 24023 server_base.cc:1034] running on GCE node
I20260619 16:58:35.590351 24023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:35.590701 24023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:35.591897 24023 hybrid_clock.cc:648] HybridClock initialized: now 1781888315591874 us; error 32 us; skew 500 ppm
Jun 19 16:58:35 dist-test-slave-vnbj krb5kdc[23782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888315, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:35.594451 24023 init.cc:377] Logged in from keytab as kudu/127.20.231.130@KRBTEST.COM (short username kudu)
I20260619 16:58:35.594993 24023 webserver.cc:460] Webserver started at http://127.20.231.130:45307/ using document root <none> and password file <none>
I20260619 16:58:35.595243 24023 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:35.595289 24023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:35.595430 24023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:35.596496 24023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-1/data/instance:
uuid: "b1d306d8d6764b31aa774e102ad0ac39"
format_stamp: "Formatted at 2026-06-19 16:58:35 on dist-test-slave-vnbj"
server_key: "45bf3456143968bf5d70cc928ab665ad"
server_key_iv: "3f495b361ac701ff33bffab2a86fd3b7"
server_key_version: "encryptionkey@0"
I20260619 16:58:35.596901 24023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance:
uuid: "b1d306d8d6764b31aa774e102ad0ac39"
format_stamp: "Formatted at 2026-06-19 16:58:35 on dist-test-slave-vnbj"
server_key: "45bf3456143968bf5d70cc928ab665ad"
server_key_iv: "3f495b361ac701ff33bffab2a86fd3b7"
server_key_version: "encryptionkey@0"
I20260619 16:58:35.598244 24023 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:58:35.599069 24038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:35.599252 24023 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:35.599373 24023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-1/wal
uuid: "b1d306d8d6764b31aa774e102ad0ac39"
format_stamp: "Formatted at 2026-06-19 16:58:35 on dist-test-slave-vnbj"
server_key: "45bf3456143968bf5d70cc928ab665ad"
server_key_iv: "3f495b361ac701ff33bffab2a86fd3b7"
server_key_version: "encryptionkey@0"
I20260619 16:58:35.599433 24023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:35.613338 24023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:35.615314 24023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:35.615428 24023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:35.615608 24023 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:58:35.615933 24023 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:35.615967 24023 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:35.616012 24023 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:35.616040 24023 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:35.622236 24023 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.130:43369
I20260619 16:58:35.622278 24151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.130:43369 every 8 connection(s)
I20260619 16:58:35.622612 24023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
Jun 19 16:58:35 dist-test-slave-vnbj krb5kdc[23782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888315, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:35.629511 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 24023
I20260619 16:58:35.629616 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance
I20260619 16:58:35.629894 21406 external_mini_cluster.cc:1428] Setting key 6f951e7c3e134295775ae6b8a09c4f87
I20260619 16:58:35.633387 23878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:35.623408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:51695 (local address 127.20.231.190:44201)
0619 16:58:35.623469 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:35.623471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:35.623684 (+ 213us) server_negotiation.cc:407] Connection header received
0619 16:58:35.624277 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:35.624278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:35.624291 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:35.624347 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:35.625356 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:35.625855 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:35.627072 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:35.627342 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:35.627428 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:35.629816 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:35.629817 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:35.629818 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:35.629826 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:35.631567 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:35.632131 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:35.632133 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:35.632133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:35.632199 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:35.632511 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:35.632512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:35.632513 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:35.632678 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:35.632758 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:35.633139 (+ 381us) server_negotiation.cc:299] Negotiation successful
0619 16:58:35.633235 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 16:58:35.633586 24154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:35.623393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:44201 (local address 127.20.231.130:51695)
0619 16:58:35.623564 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:35.623601 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:35.624146 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:35.624418 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:35.624422 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:35.625012 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:35.625209 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:35.625215 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:35.625958 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:35.625959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:35.626978 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:35.626981 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:35.627437 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:35.627438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:35.627580 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:35.628198 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:35.628207 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:35.629707 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:35.631661 (+ 1954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:35.631669 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:35.631672 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:35.632021 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:35.632277 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:35.632277 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:35.632278 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:35.632430 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:35.632762 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:35.632769 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:35.632968 (+ 199us) client_negotiation.cc:769] Sending connection context
0619 16:58:35.633098 (+ 130us) client_negotiation.cc:240] Negotiation successful
0619 16:58:35.633295 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":77,"threads_started":1}
I20260619 16:58:35.634205 24152 heartbeater.cc:347] Connected to a master server at 127.20.231.190:44201
I20260619 16:58:35.634325 24152 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:35.634541 24152 heartbeater.cc:511] Master 127.20.231.190:44201 requested a full tablet report, sending...
I20260619 16:58:35.635066 23827 ts_manager.cc:194] Registered new tserver with Master: b1d306d8d6764b31aa774e102ad0ac39 (127.20.231.130:43369)
I20260619 16:58:35.635641 23827 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.130@KRBTEST.COM'} at 127.20.231.130:51695
WARNING: no policy specified for kudu/127.20.231.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab.
Entry for principal kudu/127.20.231.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab.
I20260619 16:58:35.658206 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.131:0
--local_ip_for_outbound_sockets=127.20.231.131
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:44201
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:39445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:35.745150 24157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:35.745445 24157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:35.745489 24157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:35.747607 24157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:35.747689 24157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:35.747741 24157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.131
I20260619 16:58:35.749397 24157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:39445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:44201
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.131
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:35.749737 24157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:35.749997 24157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:35.755822 24165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:35.755841 24162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:35.755976 24163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:35.756250 24157 server_base.cc:1034] running on GCE node
I20260619 16:58:35.756448 24157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:35.756743 24157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:35.757925 24157 hybrid_clock.cc:648] HybridClock initialized: now 1781888315757909 us; error 29 us; skew 500 ppm
Jun 19 16:58:35 dist-test-slave-vnbj krb5kdc[23782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888315, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:35.761168 24157 init.cc:377] Logged in from keytab as kudu/127.20.231.131@KRBTEST.COM (short username kudu)
I20260619 16:58:35.761639 24157 webserver.cc:460] Webserver started at http://127.20.231.131:46399/ using document root <none> and password file <none>
I20260619 16:58:35.761891 24157 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:35.761965 24157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:35.762115 24157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:35.763366 24157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-2/data/instance:
uuid: "18ff484725e2408fa2f692079bfdd574"
format_stamp: "Formatted at 2026-06-19 16:58:35 on dist-test-slave-vnbj"
server_key: "a1040a3952634ba5ee39af96a230ce34"
server_key_iv: "95d3c9b66253cf60b89d897ee7c25056"
server_key_version: "encryptionkey@0"
I20260619 16:58:35.763741 24157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance:
uuid: "18ff484725e2408fa2f692079bfdd574"
format_stamp: "Formatted at 2026-06-19 16:58:35 on dist-test-slave-vnbj"
server_key: "a1040a3952634ba5ee39af96a230ce34"
server_key_iv: "95d3c9b66253cf60b89d897ee7c25056"
server_key_version: "encryptionkey@0"
I20260619 16:58:35.765313 24157 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:58:35.766163 24172 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:35.766371 24157 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:35.766500 24157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-2/wal
uuid: "18ff484725e2408fa2f692079bfdd574"
format_stamp: "Formatted at 2026-06-19 16:58:35 on dist-test-slave-vnbj"
server_key: "a1040a3952634ba5ee39af96a230ce34"
server_key_iv: "95d3c9b66253cf60b89d897ee7c25056"
server_key_version: "encryptionkey@0"
I20260619 16:58:35.766575 24157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:35.778087 24157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:35.780387 24157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:35.780503 24157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:35.780746 24157 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:58:35.781019 24157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:35.781049 24157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:35.781073 24157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:35.781085 24157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:35.787420 24157 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.131:36311
I20260619 16:58:35.787489 24285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.131:36311 every 8 connection(s)
I20260619 16:58:35.787827 24157 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
I20260619 16:58:35.793699 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 24157
I20260619 16:58:35.793802 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance
I20260619 16:58:35.793982 21406 external_mini_cluster.cc:1428] Setting key 8b2e20137849618fc41385bc881ae41e
Jun 19 16:58:35 dist-test-slave-vnbj krb5kdc[23782](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888315, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:35.797561 23878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:35.788595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:40041 (local address 127.20.231.190:44201)
0619 16:58:35.788658 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:35.788660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:35.788864 (+ 204us) server_negotiation.cc:407] Connection header received
0619 16:58:35.789800 (+ 936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:35.789801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:35.789811 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:35.789871 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:35.790462 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:35.791012 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:35.792156 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:35.792364 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:35.792424 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:35.795069 (+ 2645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:35.795070 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:35.795070 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:35.795077 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:35.796310 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:35.796695 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:35.796696 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:35.796697 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:35.796751 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:35.797024 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:35.797024 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:35.797024 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:35.797142 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:35.797197 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:35.797436 (+ 239us) server_negotiation.cc:299] Negotiation successful
0619 16:58:35.797498 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 16:58:35.797829 24288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:35.788581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:44201 (local address 127.20.231.131:40041)
0619 16:58:35.788727 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:35.788770 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:35.789704 (+ 934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:35.789925 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:35.789929 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:35.790185 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:35.790388 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:35.790393 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:35.791097 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:35.791098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:35.792063 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:35.792065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:35.792446 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:35.792447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:35.792565 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:35.793119 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:35.793127 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:35.794962 (+ 1835us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:35.796387 (+ 1425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:35.796392 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:35.796394 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:35.796608 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:35.796812 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:35.796813 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:35.796813 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:35.796917 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:35.797202 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:35.797206 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:35.797339 (+ 133us) client_negotiation.cc:769] Sending connection context
0619 16:58:35.797422 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 16:58:35.797541 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":76,"threads_started":1}
I20260619 16:58:35.798274 24286 heartbeater.cc:347] Connected to a master server at 127.20.231.190:44201
I20260619 16:58:35.798355 24286 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:35.798553 24286 heartbeater.cc:511] Master 127.20.231.190:44201 requested a full tablet report, sending...
I20260619 16:58:35.798964 23827 ts_manager.cc:194] Registered new tserver with Master: 18ff484725e2408fa2f692079bfdd574 (127.20.231.131:36311)
I20260619 16:58:35.799341 23827 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.131@KRBTEST.COM'} at 127.20.231.131:40041
I20260619 16:58:35.806114 21406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 16:58:35.817675 23878 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:35.816699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35960 (local address 127.20.231.190:44201)
0619 16:58:35.816792 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:35.816795 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:35.816895 (+ 100us) server_negotiation.cc:407] Connection header received
0619 16:58:35.816943 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:35.816944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:35.816954 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:35.817006 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:35.817593 (+ 587us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35960: BlockingRecv error: recv got EOF from 127.0.0.1:35960 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 16:58:35.818580 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 23889
I20260619 16:58:35.823076 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 24023
I20260619 16:58:35.827546 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 24157
I20260619 16:58:35.831965 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 23795
2026-06-19T16:58:35Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1911 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:58:35 dist-test-slave-vnbj krb5kdc[24298](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:58:35 dist-test-slave-vnbj krb5kdc[24298](info): set up 2 sockets
Jun 19 16:58:35 dist-test-slave-vnbj krb5kdc[24298](info): commencing operation
krb5kdc: starting...
W20260619 16:58:36.874826 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 16:58:36 dist-test-slave-vnbj krb5kdc[24298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:58:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:58:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.231.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
I20260619 16:58:36.997769 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:39179
--webserver_interface=127.20.231.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:33307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:39179
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:37.079754 24311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:37.080029 24311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:37.080066 24311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:37.081377 24311 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:37.081458 24311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:37.081485 24311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:37.081504 24311 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:37.081534 24311 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:58:37.083171 24311 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:33307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:39179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:39179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:37.083488 24311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:37.083727 24311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:37.088835 24317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:37.088862 24316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:37.088876 24319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:37.088985 24311 server_base.cc:1034] running on GCE node
I20260619 16:58:37.089375 24311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:37.089699 24311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:37.090936 24311 hybrid_clock.cc:648] HybridClock initialized: now 1781888317090914 us; error 42 us; skew 500 ppm
Jun 19 16:58:37 dist-test-slave-vnbj krb5kdc[24298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888317, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:37.093530 24311 init.cc:377] Logged in from keytab as kudu/127.20.231.190@KRBTEST.COM (short username kudu)
I20260619 16:58:37.094013 24311 webserver.cc:460] Webserver started at http://127.20.231.190:33601/ using document root <none> and password file <none>
I20260619 16:58:37.094228 24311 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:37.094271 24311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:37.094362 24311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:37.095507 24311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "f5677c50d33048c5abf76c0eba85a844"
format_stamp: "Formatted at 2026-06-19 16:58:37 on dist-test-slave-vnbj"
server_key: "46a8909d54eda88a4268b794d6c567ae"
server_key_iv: "30d3ea823d23f0d7c68154555c862df0"
server_key_version: "encryptionkey@0"
I20260619 16:58:37.095808 24311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "f5677c50d33048c5abf76c0eba85a844"
format_stamp: "Formatted at 2026-06-19 16:58:37 on dist-test-slave-vnbj"
server_key: "46a8909d54eda88a4268b794d6c567ae"
server_key_iv: "30d3ea823d23f0d7c68154555c862df0"
server_key_version: "encryptionkey@0"
I20260619 16:58:37.097183 24311 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:58:37.097935 24326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:37.098214 24311 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:37.098330 24311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "f5677c50d33048c5abf76c0eba85a844"
format_stamp: "Formatted at 2026-06-19 16:58:37 on dist-test-slave-vnbj"
server_key: "46a8909d54eda88a4268b794d6c567ae"
server_key_iv: "30d3ea823d23f0d7c68154555c862df0"
server_key_version: "encryptionkey@0"
I20260619 16:58:37.098409 24311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:37.108860 24311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:37.111027 24311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:37.111194 24311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:37.115466 24311 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:39179
I20260619 16:58:37.115509 24378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:39179 every 8 connection(s)
I20260619 16:58:37.116034 24311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 16:58:37.116793 24379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:37.119220 24379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5677c50d33048c5abf76c0eba85a844: Bootstrap starting.
I20260619 16:58:37.119853 24379 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f5677c50d33048c5abf76c0eba85a844: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:37.120163 24379 log.cc:824] T 00000000000000000000000000000000 P f5677c50d33048c5abf76c0eba85a844: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:37.120939 24379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f5677c50d33048c5abf76c0eba85a844: No bootstrap required, opened a new log
I20260619 16:58:37.122560 24379 raft_consensus.cc:348] T 00000000000000000000000000000000 P f5677c50d33048c5abf76c0eba85a844 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5677c50d33048c5abf76c0eba85a844" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 39179 } }
I20260619 16:58:37.122730 24379 raft_consensus.cc:374] T 00000000000000000000000000000000 P f5677c50d33048c5abf76c0eba85a844 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:37.122741 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 24311
I20260619 16:58:37.122774 24379 raft_consensus.cc:729] T 00000000000000000000000000000000 P f5677c50d33048c5abf76c0eba85a844 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5677c50d33048c5abf76c0eba85a844, State: Initialized, Role: FOLLOWER
I20260619 16:58:37.122896 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 16:58:37.123015 24379 consensus_queue.cc:260] T 00000000000000000000000000000000 P f5677c50d33048c5abf76c0eba85a844 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5677c50d33048c5abf76c0eba85a844" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 39179 } }
I20260619 16:58:37.123080 21406 external_mini_cluster.cc:1428] Setting key 6c82bab77ec782a068429dbefcef4d84
I20260619 16:58:37.123122 24379 raft_consensus.cc:388] T 00000000000000000000000000000000 P f5677c50d33048c5abf76c0eba85a844 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:58:37.123162 24379 raft_consensus.cc:482] T 00000000000000000000000000000000 P f5677c50d33048c5abf76c0eba85a844 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:58:37.123219 24379 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f5677c50d33048c5abf76c0eba85a844 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:37.124554 24379 raft_consensus.cc:504] T 00000000000000000000000000000000 P f5677c50d33048c5abf76c0eba85a844 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5677c50d33048c5abf76c0eba85a844" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 39179 } }
I20260619 16:58:37.124645 24379 leader_election.cc:302] T 00000000000000000000000000000000 P f5677c50d33048c5abf76c0eba85a844 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f5677c50d33048c5abf76c0eba85a844; no voters:
I20260619 16:58:37.124807 24379 leader_election.cc:288] T 00000000000000000000000000000000 P f5677c50d33048c5abf76c0eba85a844 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:58:37.124905 24384 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f5677c50d33048c5abf76c0eba85a844 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:37.125056 24379 sys_catalog.cc:564] T 00000000000000000000000000000000 P f5677c50d33048c5abf76c0eba85a844 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:37.125195 24384 raft_consensus.cc:686] T 00000000000000000000000000000000 P f5677c50d33048c5abf76c0eba85a844 [term 1 LEADER]: Becoming Leader. State: Replica: f5677c50d33048c5abf76c0eba85a844, State: Running, Role: LEADER
I20260619 16:58:37.125304 24384 consensus_queue.cc:237] T 00000000000000000000000000000000 P f5677c50d33048c5abf76c0eba85a844 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5677c50d33048c5abf76c0eba85a844" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 39179 } }
I20260619 16:58:37.125764 24385 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5677c50d33048c5abf76c0eba85a844 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f5677c50d33048c5abf76c0eba85a844" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5677c50d33048c5abf76c0eba85a844" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 39179 } } }
I20260619 16:58:37.125855 24385 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5677c50d33048c5abf76c0eba85a844 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:37.125983 24385 sys_catalog.cc:455] T 00000000000000000000000000000000 P f5677c50d33048c5abf76c0eba85a844 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f5677c50d33048c5abf76c0eba85a844. Latest consensus state: current_term: 1 leader_uuid: "f5677c50d33048c5abf76c0eba85a844" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f5677c50d33048c5abf76c0eba85a844" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 39179 } } }
I20260619 16:58:37.126031 24385 sys_catalog.cc:458] T 00000000000000000000000000000000 P f5677c50d33048c5abf76c0eba85a844 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:37.126138 24393 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:58:37.126777 24393 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:58:37.128316 24393 catalog_manager.cc:1347] Generated new cluster ID: 6cb2c828f1b74cf88dd90e1000d33657
I20260619 16:58:37.128368 24393 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 16:58:37 dist-test-slave-vnbj krb5kdc[24298](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888316, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:37.135546 24383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:37.123570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39480 (local address 127.20.231.190:39179)
0619 16:58:37.123892 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:37.123906 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:37.123951 (+ 45us) server_negotiation.cc:407] Connection header received
0619 16:58:37.124109 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:37.124116 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:37.124185 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:37.124338 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:37.125308 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:37.126733 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:37.127877 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:37.128451 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:37.128573 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:37.131630 (+ 3057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:37.131634 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:37.131635 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:37.131647 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:37.133672 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:37.134141 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:37.134145 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:37.134145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:37.134276 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:37.134501 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:37.134502 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:37.134502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:37.134805 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:37.134989 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:37.135100 (+ 111us) server_negotiation.cc:299] Negotiation successful
0619 16:58:37.135333 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":192,"threads_started":1}
I20260619 16:58:37.136510 21406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 16:58:37.144366 24393 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:58:37.144896 24393 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:58:37.153568 24393 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f5677c50d33048c5abf76c0eba85a844: Generated new TSK 0
I20260619 16:58:37.153792 24393 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:58:37.157110 24383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:37.152628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39492 (local address 127.20.231.190:39179)
0619 16:58:37.152674 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:37.152677 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:37.152855 (+ 178us) server_negotiation.cc:407] Connection header received
0619 16:58:37.152980 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:37.152982 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:37.153006 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:37.153074 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:37.153544 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:37.154178 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:37.155511 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:37.155741 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:37.155820 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:37.156574 (+ 754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:37.156575 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:37.156575 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:58:37.156822 (+ 247us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:37.156840 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:58:37.156846 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:37.156896 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:37.156992 (+ 96us) server_negotiation.cc:299] Negotiation successful
0619 16:58:37.157016 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 16:58:37.160007 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 24311
2026-06-19T16:58:37Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1325 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:58:37 dist-test-slave-vnbj krb5kdc[24412](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:58:37 dist-test-slave-vnbj krb5kdc[24412](info): set up 2 sockets
Jun 19 16:58:37 dist-test-slave-vnbj krb5kdc[24412](info): commencing operation
krb5kdc: starting...
W20260619 16:58:38.211177 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 16:58:38 dist-test-slave-vnbj krb5kdc[24412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:58:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:58:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.231.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
I20260619 16:58:38.334067 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:46325
--webserver_interface=127.20.231.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:35411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:46325
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:38.414677 24425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:38.414920 24425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:38.414943 24425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:38.416155 24425 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:38.416201 24425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:38.416216 24425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:38.416229 24425 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:38.416241 24425 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:58:38.417788 24425 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:35411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:46325
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:46325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:38.418027 24425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:38.418294 24425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:38.423444 24430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:38.423535 24425 server_base.cc:1034] running on GCE node
W20260619 16:58:38.423466 24433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:38.423477 24431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:38.423942 24425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:38.424238 24425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:38.425410 24425 hybrid_clock.cc:648] HybridClock initialized: now 1781888318425399 us; error 30 us; skew 500 ppm
Jun 19 16:58:38 dist-test-slave-vnbj krb5kdc[24412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888318, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:38.428092 24425 init.cc:377] Logged in from keytab as kudu/127.20.231.190@KRBTEST.COM (short username kudu)
I20260619 16:58:38.428591 24425 webserver.cc:460] Webserver started at http://127.20.231.190:34811/ using document root <none> and password file <none>
I20260619 16:58:38.428834 24425 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:38.428905 24425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:38.429056 24425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:38.430157 24425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "d0903997816e44ea9ab303d2c2f51a35"
format_stamp: "Formatted at 2026-06-19 16:58:38 on dist-test-slave-vnbj"
server_key: "5b0accee255a0729c319657f96e3c3c1"
server_key_iv: "aa19767a5b75b892e2de68c0a267d6f9"
server_key_version: "encryptionkey@0"
I20260619 16:58:38.430544 24425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "d0903997816e44ea9ab303d2c2f51a35"
format_stamp: "Formatted at 2026-06-19 16:58:38 on dist-test-slave-vnbj"
server_key: "5b0accee255a0729c319657f96e3c3c1"
server_key_iv: "aa19767a5b75b892e2de68c0a267d6f9"
server_key_version: "encryptionkey@0"
I20260619 16:58:38.432034 24425 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:58:38.432834 24440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:38.433081 24425 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:38.433193 24425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "d0903997816e44ea9ab303d2c2f51a35"
format_stamp: "Formatted at 2026-06-19 16:58:38 on dist-test-slave-vnbj"
server_key: "5b0accee255a0729c319657f96e3c3c1"
server_key_iv: "aa19767a5b75b892e2de68c0a267d6f9"
server_key_version: "encryptionkey@0"
I20260619 16:58:38.433269 24425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:38.445801 24425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:38.447866 24425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:38.448038 24425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:38.452505 24425 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:46325
I20260619 16:58:38.452541 24492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:46325 every 8 connection(s)
I20260619 16:58:38.452898 24425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 16:58:38.453871 24493 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:38.456266 24493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0903997816e44ea9ab303d2c2f51a35: Bootstrap starting.
I20260619 16:58:38.456835 24493 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d0903997816e44ea9ab303d2c2f51a35: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:38.457110 24493 log.cc:824] T 00000000000000000000000000000000 P d0903997816e44ea9ab303d2c2f51a35: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:38.457945 24493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d0903997816e44ea9ab303d2c2f51a35: No bootstrap required, opened a new log
I20260619 16:58:38.459241 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 24425
I20260619 16:58:38.459376 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 16:58:38.459579 21406 external_mini_cluster.cc:1428] Setting key 7120e6c40f702d03e9334f55bcc9e9eb
I20260619 16:58:38.460489 24493 raft_consensus.cc:348] T 00000000000000000000000000000000 P d0903997816e44ea9ab303d2c2f51a35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0903997816e44ea9ab303d2c2f51a35" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 46325 } }
I20260619 16:58:38.460676 24493 raft_consensus.cc:374] T 00000000000000000000000000000000 P d0903997816e44ea9ab303d2c2f51a35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:38.460727 24493 raft_consensus.cc:729] T 00000000000000000000000000000000 P d0903997816e44ea9ab303d2c2f51a35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d0903997816e44ea9ab303d2c2f51a35, State: Initialized, Role: FOLLOWER
I20260619 16:58:38.460877 24493 consensus_queue.cc:260] T 00000000000000000000000000000000 P d0903997816e44ea9ab303d2c2f51a35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0903997816e44ea9ab303d2c2f51a35" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 46325 } }
I20260619 16:58:38.460988 24493 raft_consensus.cc:388] T 00000000000000000000000000000000 P d0903997816e44ea9ab303d2c2f51a35 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:58:38.461048 24493 raft_consensus.cc:482] T 00000000000000000000000000000000 P d0903997816e44ea9ab303d2c2f51a35 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:58:38.461110 24493 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d0903997816e44ea9ab303d2c2f51a35 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:38.462080 24493 raft_consensus.cc:504] T 00000000000000000000000000000000 P d0903997816e44ea9ab303d2c2f51a35 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0903997816e44ea9ab303d2c2f51a35" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 46325 } }
I20260619 16:58:38.462227 24493 leader_election.cc:302] T 00000000000000000000000000000000 P d0903997816e44ea9ab303d2c2f51a35 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d0903997816e44ea9ab303d2c2f51a35; no voters:
I20260619 16:58:38.462432 24493 leader_election.cc:288] T 00000000000000000000000000000000 P d0903997816e44ea9ab303d2c2f51a35 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:58:38.462472 24498 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d0903997816e44ea9ab303d2c2f51a35 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:38.462633 24498 raft_consensus.cc:686] T 00000000000000000000000000000000 P d0903997816e44ea9ab303d2c2f51a35 [term 1 LEADER]: Becoming Leader. State: Replica: d0903997816e44ea9ab303d2c2f51a35, State: Running, Role: LEADER
I20260619 16:58:38.462718 24498 consensus_queue.cc:237] T 00000000000000000000000000000000 P d0903997816e44ea9ab303d2c2f51a35 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0903997816e44ea9ab303d2c2f51a35" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 46325 } }
I20260619 16:58:38.462754 24493 sys_catalog.cc:564] T 00000000000000000000000000000000 P d0903997816e44ea9ab303d2c2f51a35 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:38.463061 24498 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0903997816e44ea9ab303d2c2f51a35 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d0903997816e44ea9ab303d2c2f51a35. Latest consensus state: current_term: 1 leader_uuid: "d0903997816e44ea9ab303d2c2f51a35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0903997816e44ea9ab303d2c2f51a35" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 46325 } } }
I20260619 16:58:38.463137 24498 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0903997816e44ea9ab303d2c2f51a35 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:38.463306 24499 sys_catalog.cc:455] T 00000000000000000000000000000000 P d0903997816e44ea9ab303d2c2f51a35 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d0903997816e44ea9ab303d2c2f51a35" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d0903997816e44ea9ab303d2c2f51a35" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 46325 } } }
I20260619 16:58:38.463388 24499 sys_catalog.cc:458] T 00000000000000000000000000000000 P d0903997816e44ea9ab303d2c2f51a35 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:38.463510 24502 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:58:38.464186 24502 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:58:38.465770 24502 catalog_manager.cc:1347] Generated new cluster ID: 783cca84ae514ff098748a1dccab8781
I20260619 16:58:38.465822 24502 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 16:58:38 dist-test-slave-vnbj krb5kdc[24412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888318, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:38.473716 24497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:38.460159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39076 (local address 127.20.231.190:46325)
0619 16:58:38.460358 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:38.460365 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:38.460391 (+ 26us) server_negotiation.cc:407] Connection header received
0619 16:58:38.460487 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:38.460492 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:38.460534 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:38.460667 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:38.461397 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:38.462313 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:38.463677 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:38.464435 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:38.464536 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:38.469637 (+ 5101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:38.469641 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:38.469641 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:38.469654 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:38.471722 (+ 2068us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:38.472196 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:38.472199 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:38.472199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:38.472328 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:38.472572 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:38.472573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:38.472573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:38.472876 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:38.473024 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:38.473200 (+ 176us) server_negotiation.cc:299] Negotiation successful
0619 16:58:38.473351 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":98,"threads_started":1}
I20260619 16:58:38.475885 24502 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:58:38.476406 24502 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:58:38.482903 24502 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d0903997816e44ea9ab303d2c2f51a35: Generated new TSK 0
I20260619 16:58:38.483088 24502 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.231.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab.
Entry for principal kudu/127.20.231.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab.
I20260619 16:58:38.509761 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.129:0
--local_ip_for_outbound_sockets=127.20.231.129
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:46325
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:35411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:38.591761 24519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:38.592010 24519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:38.592049 24519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:38.593240 24519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:38.593323 24519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:38.593370 24519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.129
I20260619 16:58:38.594980 24519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:35411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:46325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.129
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:38.595264 24519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:38.595522 24519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:38.600728 24524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:38.600760 24527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:38.600745 24525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:38.601006 24519 server_base.cc:1034] running on GCE node
I20260619 16:58:38.601256 24519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:38.601521 24519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:38.602689 24519 hybrid_clock.cc:648] HybridClock initialized: now 1781888318602678 us; error 34 us; skew 500 ppm
Jun 19 16:58:38 dist-test-slave-vnbj krb5kdc[24412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888318, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:38.605363 24519 init.cc:377] Logged in from keytab as kudu/127.20.231.129@KRBTEST.COM (short username kudu)
I20260619 16:58:38.605808 24519 webserver.cc:460] Webserver started at http://127.20.231.129:46553/ using document root <none> and password file <none>
I20260619 16:58:38.606057 24519 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:38.606127 24519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:38.606281 24519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:38.607401 24519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-0/data/instance:
uuid: "b76b09ad272e4128810b541da75a0f33"
format_stamp: "Formatted at 2026-06-19 16:58:38 on dist-test-slave-vnbj"
server_key: "6b647165dc0c9354fbe6e75599a5e1cf"
server_key_iv: "d295e7dedcf9936419dfb24b7201f976"
server_key_version: "encryptionkey@0"
I20260619 16:58:38.607764 24519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance:
uuid: "b76b09ad272e4128810b541da75a0f33"
format_stamp: "Formatted at 2026-06-19 16:58:38 on dist-test-slave-vnbj"
server_key: "6b647165dc0c9354fbe6e75599a5e1cf"
server_key_iv: "d295e7dedcf9936419dfb24b7201f976"
server_key_version: "encryptionkey@0"
I20260619 16:58:38.609091 24519 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:58:38.609807 24534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:38.609974 24519 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:58:38.610074 24519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-0/wal
uuid: "b76b09ad272e4128810b541da75a0f33"
format_stamp: "Formatted at 2026-06-19 16:58:38 on dist-test-slave-vnbj"
server_key: "6b647165dc0c9354fbe6e75599a5e1cf"
server_key_iv: "d295e7dedcf9936419dfb24b7201f976"
server_key_version: "encryptionkey@0"
I20260619 16:58:38.610157 24519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:38.630756 24519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:38.633132 24519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:38.633323 24519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:38.633594 24519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:58:38.633937 24519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:38.633994 24519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:38.634032 24519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:38.634078 24519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:38.640477 24519 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.129:46169
I20260619 16:58:38.640520 24647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.129:46169 every 8 connection(s)
I20260619 16:58:38.640910 24519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
I20260619 16:58:38.645256 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 24519
I20260619 16:58:38.645378 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance
I20260619 16:58:38.645562 21406 external_mini_cluster.cc:1428] Setting key 414e5b4ff626b97ed1cccd7fb38fcbe5
Jun 19 16:58:38 dist-test-slave-vnbj krb5kdc[24412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888318, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:38.652902 24497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:38.641781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:36943 (local address 127.20.231.190:46325)
0619 16:58:38.641903 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:38.641905 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:38.641942 (+ 37us) server_negotiation.cc:407] Connection header received
0619 16:58:38.642492 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:38.642493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:38.642508 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:38.642563 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:38.643656 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:38.644156 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:38.645263 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:38.645602 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:38.645680 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:38.649036 (+ 3356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:38.649037 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:38.649037 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:38.649048 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:38.650971 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:38.651657 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:38.651658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:38.651658 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:38.651773 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:38.652154 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:38.652155 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:38.652155 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:38.652274 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:38.652337 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:38.652744 (+ 407us) server_negotiation.cc:299] Negotiation successful
0619 16:58:38.652835 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260619 16:58:38.653280 24650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:38.641656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:46325 (local address 127.20.231.129:36943)
0619 16:58:38.641836 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:38.641874 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:38.642386 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:38.642617 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:38.642620 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:38.643374 (+ 754us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:38.643575 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:38.643580 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:38.644269 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:38.644270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:38.645174 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:38.645176 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:38.645876 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:38.645877 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:38.646062 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:38.646781 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:38.646801 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:38.648910 (+ 2109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:38.651070 (+ 2160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:38.651080 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:38.651082 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:38.651494 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:38.651862 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:38.651863 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:38.651864 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:38.652048 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:38.652357 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:38.652364 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:38.652576 (+ 212us) client_negotiation.cc:769] Sending connection context
0619 16:58:38.652732 (+ 156us) client_negotiation.cc:240] Negotiation successful
0619 16:58:38.652947 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":99,"threads_started":1}
I20260619 16:58:38.653988 24648 heartbeater.cc:347] Connected to a master server at 127.20.231.190:46325
I20260619 16:58:38.654124 24648 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:38.654361 24648 heartbeater.cc:511] Master 127.20.231.190:46325 requested a full tablet report, sending...
I20260619 16:58:38.655020 24457 ts_manager.cc:194] Registered new tserver with Master: b76b09ad272e4128810b541da75a0f33 (127.20.231.129:46169)
WARNING: no policy specified for kudu/127.20.231.130@KRBTEST.COM; defaulting to no policy
I20260619 16:58:38.656104 24457 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.129@KRBTEST.COM'} at 127.20.231.129:36943
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab.
Entry for principal kudu/127.20.231.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab.
I20260619 16:58:38.674147 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.130:0
--local_ip_for_outbound_sockets=127.20.231.130
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:46325
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:35411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:38.755929 24653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:38.756143 24653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:38.756167 24653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:38.757262 24653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:38.757299 24653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:38.757332 24653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.130
I20260619 16:58:38.758802 24653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:35411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:46325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.130
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:38.759030 24653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:38.759214 24653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:38.764271 24659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:38.764292 24658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:38.764292 24661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:38.764424 24653 server_base.cc:1034] running on GCE node
I20260619 16:58:38.764746 24653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:38.764978 24653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:38.766120 24653 hybrid_clock.cc:648] HybridClock initialized: now 1781888318766110 us; error 27 us; skew 500 ppm
Jun 19 16:58:38 dist-test-slave-vnbj krb5kdc[24412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888318, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:38.768707 24653 init.cc:377] Logged in from keytab as kudu/127.20.231.130@KRBTEST.COM (short username kudu)
I20260619 16:58:38.769241 24653 webserver.cc:460] Webserver started at http://127.20.231.130:43241/ using document root <none> and password file <none>
I20260619 16:58:38.769503 24653 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:38.769551 24653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:38.769708 24653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:38.770778 24653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-1/data/instance:
uuid: "251d166123e0462282166a837dd1ebd1"
format_stamp: "Formatted at 2026-06-19 16:58:38 on dist-test-slave-vnbj"
server_key: "9c970d1ad96d6b81a3eba84ded4838f9"
server_key_iv: "18244b11d51bd96f159989b04428534a"
server_key_version: "encryptionkey@0"
I20260619 16:58:38.771205 24653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance:
uuid: "251d166123e0462282166a837dd1ebd1"
format_stamp: "Formatted at 2026-06-19 16:58:38 on dist-test-slave-vnbj"
server_key: "9c970d1ad96d6b81a3eba84ded4838f9"
server_key_iv: "18244b11d51bd96f159989b04428534a"
server_key_version: "encryptionkey@0"
I20260619 16:58:38.772534 24653 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:38.773391 24668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:38.773586 24653 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:38.773702 24653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-1/wal
uuid: "251d166123e0462282166a837dd1ebd1"
format_stamp: "Formatted at 2026-06-19 16:58:38 on dist-test-slave-vnbj"
server_key: "9c970d1ad96d6b81a3eba84ded4838f9"
server_key_iv: "18244b11d51bd96f159989b04428534a"
server_key_version: "encryptionkey@0"
I20260619 16:58:38.773780 24653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:38.791729 24653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:38.793859 24653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:38.794034 24653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:38.794519 24653 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:58:38.795042 24653 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:38.795086 24653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:38.795112 24653 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:38.795157 24653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:38.801787 24653 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.130:43537
I20260619 16:58:38.801882 24781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.130:43537 every 8 connection(s)
I20260619 16:58:38.802224 24653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
Jun 19 16:58:38 dist-test-slave-vnbj krb5kdc[24412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888318, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:38.809404 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 24653
I20260619 16:58:38.809518 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance
I20260619 16:58:38.810062 21406 external_mini_cluster.cc:1428] Setting key b6bd2730f34741ab89c18267c76212d3
I20260619 16:58:38.812565 24497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:38.803088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:53101 (local address 127.20.231.190:46325)
0619 16:58:38.803166 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:38.803169 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:38.803338 (+ 169us) server_negotiation.cc:407] Connection header received
0619 16:58:38.803880 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:38.803881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:38.803892 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:38.803963 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:38.805000 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:38.805499 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:38.806793 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:38.807058 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:38.807123 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:38.809293 (+ 2170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:38.809294 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:38.809295 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:38.809303 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:38.811059 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:38.811506 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:38.811507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:38.811508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:38.811573 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:38.811832 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:38.811833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:38.811834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:38.811978 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:38.812047 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:38.812340 (+ 293us) server_negotiation.cc:299] Negotiation successful
0619 16:58:38.812431 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 16:58:38.812680 24784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:38.803040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:46325 (local address 127.20.231.130:53101)
0619 16:58:38.803204 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:38.803243 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:38.803788 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:38.804023 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:38.804026 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:38.804659 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:38.804913 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:38.804918 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:38.805601 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:38.805602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:38.806700 (+ 1098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:38.806702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:38.807171 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:38.807172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:38.807292 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:38.807770 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:38.807778 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:38.809179 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:38.811184 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:38.811190 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:38.811192 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:38.811427 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:38.811645 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:38.811645 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:38.811646 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:38.811766 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:38.812043 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:38.812048 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:38.812198 (+ 150us) client_negotiation.cc:769] Sending connection context
0619 16:58:38.812294 (+ 96us) client_negotiation.cc:240] Negotiation successful
0619 16:58:38.812467 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":96,"threads_started":1}
I20260619 16:58:38.813436 24782 heartbeater.cc:347] Connected to a master server at 127.20.231.190:46325
I20260619 16:58:38.813551 24782 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:38.813760 24782 heartbeater.cc:511] Master 127.20.231.190:46325 requested a full tablet report, sending...
I20260619 16:58:38.814232 24457 ts_manager.cc:194] Registered new tserver with Master: 251d166123e0462282166a837dd1ebd1 (127.20.231.130:43537)
I20260619 16:58:38.814656 24457 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.130@KRBTEST.COM'} at 127.20.231.130:53101
WARNING: no policy specified for kudu/127.20.231.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab.
Entry for principal kudu/127.20.231.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab.
I20260619 16:58:38.838371 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.131:0
--local_ip_for_outbound_sockets=127.20.231.131
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:46325
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:35411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:38.921446 24787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:38.921653 24787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:38.921675 24787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:38.923259 24787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:38.923337 24787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:38.923374 24787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.131
I20260619 16:58:38.925177 24787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:35411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:46325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.131
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:38.925463 24787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:38.925729 24787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:38.931232 24795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:38.931226 24793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:38.931478 24787 server_base.cc:1034] running on GCE node
W20260619 16:58:38.931521 24792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:38.931769 24787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:38.932020 24787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:38.933184 24787 hybrid_clock.cc:648] HybridClock initialized: now 1781888318933189 us; error 46 us; skew 500 ppm
Jun 19 16:58:38 dist-test-slave-vnbj krb5kdc[24412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888318, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:38.935771 24787 init.cc:377] Logged in from keytab as kudu/127.20.231.131@KRBTEST.COM (short username kudu)
I20260619 16:58:38.936352 24787 webserver.cc:460] Webserver started at http://127.20.231.131:37517/ using document root <none> and password file <none>
I20260619 16:58:38.936561 24787 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:38.936602 24787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:38.936690 24787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:38.937834 24787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-2/data/instance:
uuid: "65b263ccd3f844fc9645e4b88232d129"
format_stamp: "Formatted at 2026-06-19 16:58:38 on dist-test-slave-vnbj"
server_key: "9b5ac67428e5861b524ba017d4b948d2"
server_key_iv: "f38fb5ded7ab35b59595ed23c32d931d"
server_key_version: "encryptionkey@0"
I20260619 16:58:38.938131 24787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance:
uuid: "65b263ccd3f844fc9645e4b88232d129"
format_stamp: "Formatted at 2026-06-19 16:58:38 on dist-test-slave-vnbj"
server_key: "9b5ac67428e5861b524ba017d4b948d2"
server_key_iv: "f38fb5ded7ab35b59595ed23c32d931d"
server_key_version: "encryptionkey@0"
I20260619 16:58:38.939482 24787 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:58:38.940209 24802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:38.940462 24787 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:38.940574 24787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-2/wal
uuid: "65b263ccd3f844fc9645e4b88232d129"
format_stamp: "Formatted at 2026-06-19 16:58:38 on dist-test-slave-vnbj"
server_key: "9b5ac67428e5861b524ba017d4b948d2"
server_key_iv: "f38fb5ded7ab35b59595ed23c32d931d"
server_key_version: "encryptionkey@0"
I20260619 16:58:38.940768 24787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:38.959414 24787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:38.961932 24787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:38.962127 24787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:38.962390 24787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:58:38.962759 24787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:38.962916 24787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:38.962978 24787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:38.963001 24787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:38.969451 24787 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.131:36281
I20260619 16:58:38.969537 24915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.131:36281 every 8 connection(s)
I20260619 16:58:38.969882 24787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
I20260619 16:58:38.974103 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 24787
I20260619 16:58:38.974175 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance
I20260619 16:58:38.974352 21406 external_mini_cluster.cc:1428] Setting key b170ec5e02cfac3178618a3dfe9362f8
Jun 19 16:58:38 dist-test-slave-vnbj krb5kdc[24412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888318, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:38.980892 24497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:38.970635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:53099 (local address 127.20.231.190:46325)
0619 16:58:38.970725 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:38.970728 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:38.970955 (+ 227us) server_negotiation.cc:407] Connection header received
0619 16:58:38.971583 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:38.971584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:38.971597 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:38.971677 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:38.972750 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:38.973299 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:38.974630 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:38.974941 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:38.975040 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:38.977412 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:38.977413 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:38.977413 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:38.977421 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:38.979377 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:38.979829 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:38.979831 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:38.979832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:38.979904 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:38.980184 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:38.980185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:38.980185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:38.980319 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:38.980404 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:38.980700 (+ 296us) server_negotiation.cc:299] Negotiation successful
0619 16:58:38.980790 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 16:58:38.981128 24918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:38.970601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:46325 (local address 127.20.231.131:53099)
0619 16:58:38.970781 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:38.970833 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:38.971462 (+ 629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:38.971756 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:38.971760 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:38.972416 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:38.972632 (+ 216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:38.972638 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:38.973404 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:38.973405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:38.974484 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:38.974488 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:38.975050 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:38.975051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:38.975215 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:38.975777 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:38.975785 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:38.977236 (+ 1451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:38.979465 (+ 2229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:38.979472 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:38.979474 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:38.979730 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:38.979979 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:38.979980 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:38.979980 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:38.980109 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:38.980393 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:38.980399 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:38.980552 (+ 153us) client_negotiation.cc:769] Sending connection context
0619 16:58:38.980677 (+ 125us) client_negotiation.cc:240] Negotiation successful
0619 16:58:38.980854 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":87,"threads_started":1}
I20260619 16:58:38.981621 24916 heartbeater.cc:347] Connected to a master server at 127.20.231.190:46325
I20260619 16:58:38.981735 24916 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:38.981918 24916 heartbeater.cc:511] Master 127.20.231.190:46325 requested a full tablet report, sending...
I20260619 16:58:38.982375 24457 ts_manager.cc:194] Registered new tserver with Master: 65b263ccd3f844fc9645e4b88232d129 (127.20.231.131:36281)
I20260619 16:58:38.982757 24457 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.131@KRBTEST.COM'} at 127.20.231.131:53099
I20260619 16:58:38.986451 21406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 16:58:38 dist-test-slave-vnbj krb5kdc[24412](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888318, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Jun 19 16:58:38 dist-test-slave-vnbj krb5kdc[24412](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888318, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:39.009213 24497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:39.002334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39084 (local address 127.20.231.190:46325)
0619 16:58:39.002425 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:39.002427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:39.002461 (+ 34us) server_negotiation.cc:407] Connection header received
0619 16:58:39.002506 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:39.002507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:39.002517 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:39.002564 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:39.003213 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:39.003703 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:39.004418 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:39.004593 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:39.004642 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:39.006987 (+ 2345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:39.006994 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:39.006994 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:39.007001 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:39.008220 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:39.008534 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:39.008535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:39.008535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:39.008596 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:39.008757 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:39.008758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:39.008758 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:39.008910 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:39.008990 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:39.009084 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 16:58:39.009147 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260619 16:58:39.009495 24457 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:39084
I20260619 16:58:39.010048 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 24519
I20260619 16:58:39.014591 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 24653
I20260619 16:58:39.018743 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 24787
I20260619 16:58:39.022933 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 24425
2026-06-19T16:58:39Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1864 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:58:39 dist-test-slave-vnbj krb5kdc[24928](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:58:39 dist-test-slave-vnbj krb5kdc[24928](info): set up 2 sockets
Jun 19 16:58:39 dist-test-slave-vnbj krb5kdc[24928](info): commencing operation
krb5kdc: starting...
W20260619 16:58:40.175238 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 16:58:40 dist-test-slave-vnbj krb5kdc[24928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:58:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:58:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.231.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
I20260619 16:58:40.300207 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:38463
--webserver_interface=127.20.231.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:36549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:38463
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:40.380829 24941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:40.381076 24941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:40.381114 24941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:40.382356 24941 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:40.382432 24941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:40.382457 24941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:40.382478 24941 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:40.382506 24941 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:58:40.384037 24941 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:36549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:38463
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:38463
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:40.384325 24941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:40.384558 24941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:40.389689 24949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:40.389711 24946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:40.389689 24947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:40.389711 24941 server_base.cc:1034] running on GCE node
I20260619 16:58:40.390161 24941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:40.390403 24941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:40.391568 24941 hybrid_clock.cc:648] HybridClock initialized: now 1781888320391556 us; error 34 us; skew 500 ppm
Jun 19 16:58:40 dist-test-slave-vnbj krb5kdc[24928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888320, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:40.394186 24941 init.cc:377] Logged in from keytab as kudu/127.20.231.190@KRBTEST.COM (short username kudu)
I20260619 16:58:40.394693 24941 webserver.cc:460] Webserver started at http://127.20.231.190:34905/ using document root <none> and password file <none>
I20260619 16:58:40.395001 24941 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:40.395048 24941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:40.395202 24941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:40.396387 24941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "ca6a4d45e0b741c8acf5d505f4398eb6"
format_stamp: "Formatted at 2026-06-19 16:58:40 on dist-test-slave-vnbj"
server_key: "85d41e16ffff3c4edff7316146d3b9b1"
server_key_iv: "5218812784db12fc9e7111fd17dda184"
server_key_version: "encryptionkey@0"
I20260619 16:58:40.396781 24941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "ca6a4d45e0b741c8acf5d505f4398eb6"
format_stamp: "Formatted at 2026-06-19 16:58:40 on dist-test-slave-vnbj"
server_key: "85d41e16ffff3c4edff7316146d3b9b1"
server_key_iv: "5218812784db12fc9e7111fd17dda184"
server_key_version: "encryptionkey@0"
I20260619 16:58:40.398154 24941 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:58:40.399087 24956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:40.399279 24941 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:40.399374 24941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "ca6a4d45e0b741c8acf5d505f4398eb6"
format_stamp: "Formatted at 2026-06-19 16:58:40 on dist-test-slave-vnbj"
server_key: "85d41e16ffff3c4edff7316146d3b9b1"
server_key_iv: "5218812784db12fc9e7111fd17dda184"
server_key_version: "encryptionkey@0"
I20260619 16:58:40.399451 24941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:40.408842 24941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:40.410636 24941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:40.410794 24941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:40.414541 24941 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:38463
I20260619 16:58:40.414592 25008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:38463 every 8 connection(s)
I20260619 16:58:40.415153 24941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 16:58:40.415833 25009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:40.418097 25009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca6a4d45e0b741c8acf5d505f4398eb6: Bootstrap starting.
I20260619 16:58:40.418684 25009 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca6a4d45e0b741c8acf5d505f4398eb6: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:40.419024 25009 log.cc:824] T 00000000000000000000000000000000 P ca6a4d45e0b741c8acf5d505f4398eb6: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:40.419704 25009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca6a4d45e0b741c8acf5d505f4398eb6: No bootstrap required, opened a new log
I20260619 16:58:40.421309 25009 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca6a4d45e0b741c8acf5d505f4398eb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca6a4d45e0b741c8acf5d505f4398eb6" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 38463 } }
I20260619 16:58:40.421447 25009 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca6a4d45e0b741c8acf5d505f4398eb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:40.421471 25009 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca6a4d45e0b741c8acf5d505f4398eb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca6a4d45e0b741c8acf5d505f4398eb6, State: Initialized, Role: FOLLOWER
I20260619 16:58:40.421626 25009 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca6a4d45e0b741c8acf5d505f4398eb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca6a4d45e0b741c8acf5d505f4398eb6" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 38463 } }
I20260619 16:58:40.421702 25009 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca6a4d45e0b741c8acf5d505f4398eb6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:58:40.421725 25009 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca6a4d45e0b741c8acf5d505f4398eb6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:58:40.421793 25009 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca6a4d45e0b741c8acf5d505f4398eb6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:40.422677 25009 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca6a4d45e0b741c8acf5d505f4398eb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca6a4d45e0b741c8acf5d505f4398eb6" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 38463 } }
I20260619 16:58:40.422763 25009 leader_election.cc:302] T 00000000000000000000000000000000 P ca6a4d45e0b741c8acf5d505f4398eb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca6a4d45e0b741c8acf5d505f4398eb6; no voters:
I20260619 16:58:40.422986 25009 leader_election.cc:288] T 00000000000000000000000000000000 P ca6a4d45e0b741c8acf5d505f4398eb6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:58:40.423139 25012 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca6a4d45e0b741c8acf5d505f4398eb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:40.423264 25009 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca6a4d45e0b741c8acf5d505f4398eb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:40.423314 25012 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca6a4d45e0b741c8acf5d505f4398eb6 [term 1 LEADER]: Becoming Leader. State: Replica: ca6a4d45e0b741c8acf5d505f4398eb6, State: Running, Role: LEADER
I20260619 16:58:40.423441 25012 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca6a4d45e0b741c8acf5d505f4398eb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca6a4d45e0b741c8acf5d505f4398eb6" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 38463 } }
I20260619 16:58:40.423940 25014 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca6a4d45e0b741c8acf5d505f4398eb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca6a4d45e0b741c8acf5d505f4398eb6. Latest consensus state: current_term: 1 leader_uuid: "ca6a4d45e0b741c8acf5d505f4398eb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca6a4d45e0b741c8acf5d505f4398eb6" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 38463 } } }
I20260619 16:58:40.423957 25013 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca6a4d45e0b741c8acf5d505f4398eb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca6a4d45e0b741c8acf5d505f4398eb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca6a4d45e0b741c8acf5d505f4398eb6" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 38463 } } }
I20260619 16:58:40.424041 25014 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca6a4d45e0b741c8acf5d505f4398eb6 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:40.424075 25013 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca6a4d45e0b741c8acf5d505f4398eb6 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:40.424260 25021 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:58:40.424768 25021 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:58:40.425144 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 24941
I20260619 16:58:40.425241 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 16:58:40.425428 21406 external_mini_cluster.cc:1428] Setting key affe343cd5d51664f5dd1b4b6cf9939b
I20260619 16:58:40.426520 25021 catalog_manager.cc:1347] Generated new cluster ID: 8d924895463845e79ba4788dab78e2cb
I20260619 16:58:40.426573 25021 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 16:58:40 dist-test-slave-vnbj krb5kdc[24928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:40.438035 25032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:40.426112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60010 (local address 127.20.231.190:38463)
0619 16:58:40.426275 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:40.426285 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:40.426320 (+ 35us) server_negotiation.cc:407] Connection header received
0619 16:58:40.426466 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:40.426474 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:40.426532 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:40.426673 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:40.427531 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:40.429157 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:40.430211 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:40.430977 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:40.431111 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:40.434049 (+ 2938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:40.434054 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:40.434054 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:40.434067 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:40.436235 (+ 2168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:40.436685 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:40.436689 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:40.436689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:40.436806 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:40.437038 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:40.437039 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:40.437039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:40.437342 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:40.437505 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:40.437653 (+ 148us) server_negotiation.cc:299] Negotiation successful
0619 16:58:40.437801 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":71,"threads_started":1}
I20260619 16:58:40.440505 25021 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:58:40.441102 25021 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:58:40.446826 25021 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca6a4d45e0b741c8acf5d505f4398eb6: Generated new TSK 0
I20260619 16:58:40.447044 25021 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.231.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab.
Entry for principal kudu/127.20.231.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab.
I20260619 16:58:40.475811 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.129:0
--local_ip_for_outbound_sockets=127.20.231.129
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:38463
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:36549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:40.557382 25035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:40.557592 25035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:40.557618 25035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:40.558841 25035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:40.558916 25035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:40.558944 25035 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.129
I20260619 16:58:40.560582 25035 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:36549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:38463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.129
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:40.560844 25035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:40.561079 25035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:40.567423 25041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:40.567407 25043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:40.567404 25040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:40.567714 25035 server_base.cc:1034] running on GCE node
I20260619 16:58:40.568011 25035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:40.568284 25035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:40.569451 25035 hybrid_clock.cc:648] HybridClock initialized: now 1781888320569438 us; error 64 us; skew 500 ppm
Jun 19 16:58:40 dist-test-slave-vnbj krb5kdc[24928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888320, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:40.572270 25035 init.cc:377] Logged in from keytab as kudu/127.20.231.129@KRBTEST.COM (short username kudu)
I20260619 16:58:40.572719 25035 webserver.cc:460] Webserver started at http://127.20.231.129:40909/ using document root <none> and password file <none>
I20260619 16:58:40.572973 25035 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:40.573046 25035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:40.573194 25035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:40.574267 25035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-0/data/instance:
uuid: "c35c647cb966497aa64f9c78dc3c8f5f"
format_stamp: "Formatted at 2026-06-19 16:58:40 on dist-test-slave-vnbj"
server_key: "31dd0c1b807a9f049428a93b815459d5"
server_key_iv: "0ce9ad56a2c57f5d87024c2b71e1bfa0"
server_key_version: "encryptionkey@0"
I20260619 16:58:40.574633 25035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance:
uuid: "c35c647cb966497aa64f9c78dc3c8f5f"
format_stamp: "Formatted at 2026-06-19 16:58:40 on dist-test-slave-vnbj"
server_key: "31dd0c1b807a9f049428a93b815459d5"
server_key_iv: "0ce9ad56a2c57f5d87024c2b71e1bfa0"
server_key_version: "encryptionkey@0"
I20260619 16:58:40.576057 25035 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:58:40.576910 25050 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:40.577093 25035 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:40.577193 25035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-0/wal
uuid: "c35c647cb966497aa64f9c78dc3c8f5f"
format_stamp: "Formatted at 2026-06-19 16:58:40 on dist-test-slave-vnbj"
server_key: "31dd0c1b807a9f049428a93b815459d5"
server_key_iv: "0ce9ad56a2c57f5d87024c2b71e1bfa0"
server_key_version: "encryptionkey@0"
I20260619 16:58:40.577277 25035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:40.590314 25035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:40.592454 25035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:40.592628 25035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:40.592876 25035 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:58:40.593189 25035 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:40.593245 25035 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:40.593284 25035 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:40.593310 25035 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:40.599272 25035 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.129:43745
I20260619 16:58:40.599327 25163 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.129:43745 every 8 connection(s)
I20260619 16:58:40.599669 25035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
I20260619 16:58:40.600867 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 25035
I20260619 16:58:40.600986 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance
I20260619 16:58:40.601177 21406 external_mini_cluster.cc:1428] Setting key 1bf72631aa50b52ebe028311ab7e73ff
Jun 19 16:58:40 dist-test-slave-vnbj krb5kdc[24928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888320, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.231.130@KRBTEST.COM; defaulting to no policy
I20260619 16:58:40.610500 25032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:40.600521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:33771 (local address 127.20.231.190:38463)
0619 16:58:40.600716 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:40.600719 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:40.600833 (+ 114us) server_negotiation.cc:407] Connection header received
0619 16:58:40.602118 (+ 1285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:40.602119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:40.602159 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:40.602209 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:40.602859 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:40.603607 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:40.604652 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:40.605024 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:40.605098 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:40.607458 (+ 2360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:40.607460 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:40.607461 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:40.607471 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:40.609114 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:40.609536 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:40.609537 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:40.609537 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:40.609598 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:40.609838 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:40.609839 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:40.609840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:40.609984 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:40.610054 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:40.610298 (+ 244us) server_negotiation.cc:299] Negotiation successful
0619 16:58:40.610397 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260619 16:58:40.610635 25166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:40.600434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:38463 (local address 127.20.231.129:33771)
0619 16:58:40.600669 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:40.600723 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:40.601991 (+ 1268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:40.602279 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:40.602284 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:40.602550 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:40.602741 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:40.602746 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:40.603727 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:40.603728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:40.604566 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:40.604570 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:40.605114 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:40.605115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:40.605218 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:40.605719 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:40.605727 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:40.607356 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:40.609214 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:40.609220 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:40.609221 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:40.609458 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:40.609669 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:40.609670 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:40.609670 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:40.609764 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:40.610049 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:40.610053 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:40.610166 (+ 113us) client_negotiation.cc:769] Sending connection context
0619 16:58:40.610246 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 16:58:40.610429 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":83,"threads_started":1}
I20260619 16:58:40.611135 25164 heartbeater.cc:347] Connected to a master server at 127.20.231.190:38463
I20260619 16:58:40.611233 25164 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:40.611411 25164 heartbeater.cc:511] Master 127.20.231.190:38463 requested a full tablet report, sending...
I20260619 16:58:40.611949 24973 ts_manager.cc:194] Registered new tserver with Master: c35c647cb966497aa64f9c78dc3c8f5f (127.20.231.129:43745)
I20260619 16:58:40.612501 24973 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.129@KRBTEST.COM'} at 127.20.231.129:33771
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab.
Entry for principal kudu/127.20.231.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab.
I20260619 16:58:40.627924 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.130:0
--local_ip_for_outbound_sockets=127.20.231.130
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:38463
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:36549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:40.711846 25169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:40.712090 25169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:40.712131 25169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:40.713289 25169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:40.713356 25169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:40.713402 25169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.130
I20260619 16:58:40.715016 25169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:36549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:38463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.130
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:40.715274 25169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:40.715504 25169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:40.720669 25175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:40.720741 25174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:40.720757 25177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:40.720914 25169 server_base.cc:1034] running on GCE node
I20260619 16:58:40.721158 25169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:40.721410 25169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:40.722558 25169 hybrid_clock.cc:648] HybridClock initialized: now 1781888320722528 us; error 44 us; skew 500 ppm
Jun 19 16:58:40 dist-test-slave-vnbj krb5kdc[24928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888320, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:40.725747 25169 init.cc:377] Logged in from keytab as kudu/127.20.231.130@KRBTEST.COM (short username kudu)
I20260619 16:58:40.726219 25169 webserver.cc:460] Webserver started at http://127.20.231.130:46121/ using document root <none> and password file <none>
I20260619 16:58:40.726449 25169 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:40.726518 25169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:40.726742 25169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:40.727880 25169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-1/data/instance:
uuid: "00f00a262ce64dcbab5c651d24fa9cd9"
format_stamp: "Formatted at 2026-06-19 16:58:40 on dist-test-slave-vnbj"
server_key: "ad510e23f1d2bdd2df317499613eace6"
server_key_iv: "424bf71257f5c0f733d6b7065d26b81f"
server_key_version: "encryptionkey@0"
I20260619 16:58:40.728246 25169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance:
uuid: "00f00a262ce64dcbab5c651d24fa9cd9"
format_stamp: "Formatted at 2026-06-19 16:58:40 on dist-test-slave-vnbj"
server_key: "ad510e23f1d2bdd2df317499613eace6"
server_key_iv: "424bf71257f5c0f733d6b7065d26b81f"
server_key_version: "encryptionkey@0"
I20260619 16:58:40.729776 25169 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 16:58:40.730648 25184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:40.730896 25169 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:40.731004 25169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-1/wal
uuid: "00f00a262ce64dcbab5c651d24fa9cd9"
format_stamp: "Formatted at 2026-06-19 16:58:40 on dist-test-slave-vnbj"
server_key: "ad510e23f1d2bdd2df317499613eace6"
server_key_iv: "424bf71257f5c0f733d6b7065d26b81f"
server_key_version: "encryptionkey@0"
I20260619 16:58:40.731087 25169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:40.755856 25169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:40.758669 25169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:40.758935 25169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:40.759253 25169 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:58:40.759642 25169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:40.759714 25169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:40.759765 25169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:40.759815 25169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:40.767428 25169 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.130:42747
I20260619 16:58:40.767472 25297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.130:42747 every 8 connection(s)
I20260619 16:58:40.767880 25169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
I20260619 16:58:40.773229 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 25169
I20260619 16:58:40.773319 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance
I20260619 16:58:40.773538 21406 external_mini_cluster.cc:1428] Setting key 877b2409dbf897f8f51b5eb34b1486cc
Jun 19 16:58:40 dist-test-slave-vnbj krb5kdc[24928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888320, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:40.780145 25032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:40.768770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:41811 (local address 127.20.231.190:38463)
0619 16:58:40.768857 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:40.768860 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:40.769116 (+ 256us) server_negotiation.cc:407] Connection header received
0619 16:58:40.770127 (+ 1011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:40.770128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:40.770139 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:40.770208 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:40.770959 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:40.771503 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:40.772680 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:40.772925 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:40.773050 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:40.776279 (+ 3229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:40.776281 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:40.776281 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:40.776289 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:40.778099 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:40.778737 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:40.778738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:40.778738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:40.778803 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:40.779234 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:40.779234 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:40.779235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:40.779374 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:40.779435 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:40.779892 (+ 457us) server_negotiation.cc:299] Negotiation successful
0619 16:58:40.780035 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 16:58:40.780406 25300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:40.768735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:38463 (local address 127.20.231.130:41811)
0619 16:58:40.768946 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:40.768988 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:40.770014 (+ 1026us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:40.770270 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:40.770273 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:40.770565 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:40.770789 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:40.770794 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:40.771619 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:40.771620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:40.772556 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:40.772558 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:40.773110 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:40.773111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:40.773251 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:40.773958 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:40.773967 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:40.776163 (+ 2196us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:40.778222 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:40.778231 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:40.778234 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:40.778573 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:40.778985 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:40.778987 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:40.778987 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:40.779152 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:40.779450 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:40.779457 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:40.779676 (+ 219us) client_negotiation.cc:769] Sending connection context
0619 16:58:40.779830 (+ 154us) client_negotiation.cc:240] Negotiation successful
0619 16:58:40.780109 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":104,"threads_started":1}
I20260619 16:58:40.781044 25298 heartbeater.cc:347] Connected to a master server at 127.20.231.190:38463
I20260619 16:58:40.781248 25298 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:40.781481 25298 heartbeater.cc:511] Master 127.20.231.190:38463 requested a full tablet report, sending...
I20260619 16:58:40.781929 24973 ts_manager.cc:194] Registered new tserver with Master: 00f00a262ce64dcbab5c651d24fa9cd9 (127.20.231.130:42747)
I20260619 16:58:40.782430 24973 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.130@KRBTEST.COM'} at 127.20.231.130:41811
WARNING: no policy specified for kudu/127.20.231.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab.
Entry for principal kudu/127.20.231.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab.
I20260619 16:58:40.803232 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.131:0
--local_ip_for_outbound_sockets=127.20.231.131
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:38463
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:36549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:40.886020 25303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:40.886276 25303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:40.886317 25303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:40.887555 25303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:40.887630 25303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:40.887674 25303 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.131
I20260619 16:58:40.889211 25303 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:36549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:38463
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.131
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:40.889518 25303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:40.889770 25303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:40.894717 25309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:40.894876 25308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:40.894685 25311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:40.895130 25303 server_base.cc:1034] running on GCE node
I20260619 16:58:40.895347 25303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:40.895543 25303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:40.896687 25303 hybrid_clock.cc:648] HybridClock initialized: now 1781888320896673 us; error 26 us; skew 500 ppm
Jun 19 16:58:40 dist-test-slave-vnbj krb5kdc[24928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888320, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:40.899748 25303 init.cc:377] Logged in from keytab as kudu/127.20.231.131@KRBTEST.COM (short username kudu)
I20260619 16:58:40.900147 25303 webserver.cc:460] Webserver started at http://127.20.231.131:36525/ using document root <none> and password file <none>
I20260619 16:58:40.900343 25303 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:40.900383 25303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:40.900472 25303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:40.901604 25303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-2/data/instance:
uuid: "e0ffc64020fc4f7fa95fc2ccc8bef2f1"
format_stamp: "Formatted at 2026-06-19 16:58:40 on dist-test-slave-vnbj"
server_key: "a164b1525986c93fc4ca4c5c2c60ee56"
server_key_iv: "1feb939c26ddfb3d28a94f6f41059cc6"
server_key_version: "encryptionkey@0"
I20260619 16:58:40.901919 25303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance:
uuid: "e0ffc64020fc4f7fa95fc2ccc8bef2f1"
format_stamp: "Formatted at 2026-06-19 16:58:40 on dist-test-slave-vnbj"
server_key: "a164b1525986c93fc4ca4c5c2c60ee56"
server_key_iv: "1feb939c26ddfb3d28a94f6f41059cc6"
server_key_version: "encryptionkey@0"
I20260619 16:58:40.903384 25303 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:58:40.904191 25318 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:40.904348 25303 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:58:40.904434 25303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-2/wal
uuid: "e0ffc64020fc4f7fa95fc2ccc8bef2f1"
format_stamp: "Formatted at 2026-06-19 16:58:40 on dist-test-slave-vnbj"
server_key: "a164b1525986c93fc4ca4c5c2c60ee56"
server_key_iv: "1feb939c26ddfb3d28a94f6f41059cc6"
server_key_version: "encryptionkey@0"
I20260619 16:58:40.904488 25303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:40.921504 25303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:40.924301 25303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:40.924463 25303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:40.924734 25303 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:58:40.925102 25303 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:40.925145 25303 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:40.925175 25303 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:40.925231 25303 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:40.932165 25303 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.131:41287
I20260619 16:58:40.932204 25431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.131:41287 every 8 connection(s)
I20260619 16:58:40.932663 25303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
I20260619 16:58:40.938650 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 25303
I20260619 16:58:40.938813 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance
I20260619 16:58:40.939023 21406 external_mini_cluster.cc:1428] Setting key 8b4e9b7873ace315eee06676064ac47c
Jun 19 16:58:40 dist-test-slave-vnbj krb5kdc[24928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888320, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:40.942714 25032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:40.933444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:38723 (local address 127.20.231.190:38463)
0619 16:58:40.933528 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:40.933530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:40.933741 (+ 211us) server_negotiation.cc:407] Connection header received
0619 16:58:40.935046 (+ 1305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:40.935048 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:40.935067 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:40.935143 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:40.935903 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:40.936411 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:40.937496 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:40.937673 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:40.937728 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:40.940276 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:40.940279 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:40.940280 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:40.940291 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:40.941468 (+ 1177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:40.941828 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:40.941828 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:40.941829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:40.941911 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:40.942119 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:40.942120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:40.942120 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:40.942255 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:40.942309 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:40.942591 (+ 282us) server_negotiation.cc:299] Negotiation successful
0619 16:58:40.942650 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 16:58:40.942914 25434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:40.933409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:38463 (local address 127.20.231.131:38723)
0619 16:58:40.933604 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:40.933644 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:40.934903 (+ 1259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:40.935252 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:40.935256 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:40.935548 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:40.935812 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:40.935817 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:40.936510 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:40.936511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:40.937414 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:40.937416 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:40.937772 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:40.937773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:40.937887 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:40.938470 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:40.938480 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:40.940190 (+ 1710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:40.941539 (+ 1349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:40.941545 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:40.941546 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:40.941760 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:40.941971 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:40.941972 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:40.941972 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:40.942066 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:40.942341 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:40.942345 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:40.942462 (+ 117us) client_negotiation.cc:769] Sending connection context
0619 16:58:40.942551 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 16:58:40.942668 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":104,"threads_started":1}
I20260619 16:58:40.943403 25432 heartbeater.cc:347] Connected to a master server at 127.20.231.190:38463
I20260619 16:58:40.943518 25432 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:40.943691 25432 heartbeater.cc:511] Master 127.20.231.190:38463 requested a full tablet report, sending...
I20260619 16:58:40.944051 24973 ts_manager.cc:194] Registered new tserver with Master: e0ffc64020fc4f7fa95fc2ccc8bef2f1 (127.20.231.131:41287)
I20260619 16:58:40.944387 24973 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.131@KRBTEST.COM'} at 127.20.231.131:38723
I20260619 16:58:40.951270 21406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 16:58:40 dist-test-slave-vnbj krb5kdc[24928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 16:58:40 dist-test-slave-vnbj krb5kdc[24928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.129@KRBTEST.COM
I20260619 16:58:40.976578 25439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:40.968023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42968 (local address 127.20.231.129:43745)
0619 16:58:40.968242 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:40.968247 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:40.968258 (+ 11us) server_negotiation.cc:407] Connection header received
0619 16:58:40.968284 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:40.968286 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:40.968299 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:40.968375 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:40.968972 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:40.969997 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:40.970747 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:40.971181 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:40.971251 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:40.973552 (+ 2301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:40.973560 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:40.973561 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:40.973568 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:40.975318 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:40.975767 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:40.975771 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:40.975771 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:40.975833 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:40.976046 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:40.976046 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:40.976046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:40.976196 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:40.976315 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:40.976378 (+ 63us) server_negotiation.cc:299] Negotiation successful
0619 16:58:40.976445 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":108,"threads_started":1}
Jun 19 16:58:40 dist-test-slave-vnbj krb5kdc[24928](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:40.984862 25032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:40.978112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60024 (local address 127.20.231.190:38463)
0619 16:58:40.978181 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:40.978183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:40.978195 (+ 12us) server_negotiation.cc:407] Connection header received
0619 16:58:40.978223 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:40.978224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:40.978232 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:40.978279 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:40.978818 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:40.979360 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:40.980095 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:40.980276 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:40.980333 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:40.982654 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:40.982655 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:40.982656 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:40.982663 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:40.983902 (+ 1239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:40.984210 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:40.984211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:40.984211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:40.984259 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:40.984404 (+ 145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:40.984404 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:40.984405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:40.984536 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:40.984608 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:40.984718 (+ 110us) server_negotiation.cc:299] Negotiation successful
0619 16:58:40.984797 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260619 16:58:40.985123 24973 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60024
I20260619 16:58:40.985527 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 25035
I20260619 16:58:40.990571 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 25169
I20260619 16:58:40.994809 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 25303
I20260619 16:58:40.999502 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 24941
2026-06-19T16:58:40Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1976 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:58:41 dist-test-slave-vnbj krb5kdc[25444](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:58:41 dist-test-slave-vnbj krb5kdc[25444](info): set up 2 sockets
Jun 19 16:58:41 dist-test-slave-vnbj krb5kdc[25444](info): commencing operation
krb5kdc: starting...
W20260619 16:58:42.055547 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 16:58:42 dist-test-slave-vnbj krb5kdc[25444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:58:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:58:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.231.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
I20260619 16:58:42.181777 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:32865
--webserver_interface=127.20.231.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:35023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:32865
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:42.262236 25457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:42.262468 25457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:42.262492 25457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:42.263609 25457 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:42.263648 25457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:42.263661 25457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:42.263674 25457 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:42.263685 25457 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:58:42.265115 25457 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:35023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:32865
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:32865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:42.265326 25457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:42.265501 25457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:42.270577 25462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:42.270641 25463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:42.270577 25465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:42.271107 25457 server_base.cc:1034] running on GCE node
I20260619 16:58:42.271353 25457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:42.271601 25457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:42.272748 25457 hybrid_clock.cc:648] HybridClock initialized: now 1781888322272738 us; error 44 us; skew 500 ppm
Jun 19 16:58:42 dist-test-slave-vnbj krb5kdc[25444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888322, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:42.279768 25457 init.cc:377] Logged in from keytab as kudu/127.20.231.190@KRBTEST.COM (short username kudu)
I20260619 16:58:42.280222 25457 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:42.280319 25457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:42.280463 25457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:42.281623 25457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "fa1cccb46bb8464393cddc057ada1e6b"
format_stamp: "Formatted at 2026-06-19 16:58:42 on dist-test-slave-vnbj"
server_key: "8b0337269152d54b0100e72cf0912de1"
server_key_iv: "417c5d9f3477e638b78a215c3aaee761"
server_key_version: "encryptionkey@0"
I20260619 16:58:42.281996 25457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "fa1cccb46bb8464393cddc057ada1e6b"
format_stamp: "Formatted at 2026-06-19 16:58:42 on dist-test-slave-vnbj"
server_key: "8b0337269152d54b0100e72cf0912de1"
server_key_iv: "417c5d9f3477e638b78a215c3aaee761"
server_key_version: "encryptionkey@0"
I20260619 16:58:42.283440 25457 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:58:42.284268 25471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:42.284464 25457 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:42.284569 25457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "fa1cccb46bb8464393cddc057ada1e6b"
format_stamp: "Formatted at 2026-06-19 16:58:42 on dist-test-slave-vnbj"
server_key: "8b0337269152d54b0100e72cf0912de1"
server_key_iv: "417c5d9f3477e638b78a215c3aaee761"
server_key_version: "encryptionkey@0"
I20260619 16:58:42.284657 25457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:42.302762 25457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:42.304909 25457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:42.305083 25457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:42.309290 25457 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:32865
I20260619 16:58:42.309306 25523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:32865 every 8 connection(s)
I20260619 16:58:42.309635 25457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 16:58:42.310518 25524 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:42.312958 25524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa1cccb46bb8464393cddc057ada1e6b: Bootstrap starting.
I20260619 16:58:42.313568 25524 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa1cccb46bb8464393cddc057ada1e6b: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:42.313903 25524 log.cc:824] T 00000000000000000000000000000000 P fa1cccb46bb8464393cddc057ada1e6b: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:42.314767 25524 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa1cccb46bb8464393cddc057ada1e6b: No bootstrap required, opened a new log
I20260619 16:58:42.316917 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 25457
I20260619 16:58:42.316771 25524 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa1cccb46bb8464393cddc057ada1e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa1cccb46bb8464393cddc057ada1e6b" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 32865 } }
I20260619 16:58:42.316982 25524 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa1cccb46bb8464393cddc057ada1e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:42.317042 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 16:58:42.317026 25524 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa1cccb46bb8464393cddc057ada1e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa1cccb46bb8464393cddc057ada1e6b, State: Initialized, Role: FOLLOWER
I20260619 16:58:42.317240 21406 external_mini_cluster.cc:1428] Setting key a1291d0cbb78ff612b2acd06dabb07cb
I20260619 16:58:42.317217 25524 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa1cccb46bb8464393cddc057ada1e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa1cccb46bb8464393cddc057ada1e6b" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 32865 } }
I20260619 16:58:42.317329 25524 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa1cccb46bb8464393cddc057ada1e6b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:58:42.317378 25524 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa1cccb46bb8464393cddc057ada1e6b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:58:42.317436 25524 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa1cccb46bb8464393cddc057ada1e6b [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:42.318324 25524 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa1cccb46bb8464393cddc057ada1e6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa1cccb46bb8464393cddc057ada1e6b" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 32865 } }
I20260619 16:58:42.318471 25524 leader_election.cc:302] T 00000000000000000000000000000000 P fa1cccb46bb8464393cddc057ada1e6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa1cccb46bb8464393cddc057ada1e6b; no voters:
I20260619 16:58:42.318642 25524 leader_election.cc:288] T 00000000000000000000000000000000 P fa1cccb46bb8464393cddc057ada1e6b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:58:42.318670 25529 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa1cccb46bb8464393cddc057ada1e6b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:42.318789 25529 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa1cccb46bb8464393cddc057ada1e6b [term 1 LEADER]: Becoming Leader. State: Replica: fa1cccb46bb8464393cddc057ada1e6b, State: Running, Role: LEADER
I20260619 16:58:42.318948 25524 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa1cccb46bb8464393cddc057ada1e6b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:42.318986 25529 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa1cccb46bb8464393cddc057ada1e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa1cccb46bb8464393cddc057ada1e6b" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 32865 } }
I20260619 16:58:42.319401 25531 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa1cccb46bb8464393cddc057ada1e6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa1cccb46bb8464393cddc057ada1e6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa1cccb46bb8464393cddc057ada1e6b" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 32865 } } }
I20260619 16:58:42.319484 25531 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa1cccb46bb8464393cddc057ada1e6b [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:42.319658 25530 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa1cccb46bb8464393cddc057ada1e6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa1cccb46bb8464393cddc057ada1e6b. Latest consensus state: current_term: 1 leader_uuid: "fa1cccb46bb8464393cddc057ada1e6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa1cccb46bb8464393cddc057ada1e6b" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 32865 } } }
I20260619 16:58:42.319778 25530 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa1cccb46bb8464393cddc057ada1e6b [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:42.319957 25533 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:58:42.320554 25533 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:58:42.324103 25533 catalog_manager.cc:1347] Generated new cluster ID: 9d433185ffa146ab8ecb503d73d2541b
I20260619 16:58:42.324182 25533 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 16:58:42 dist-test-slave-vnbj krb5kdc[25444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:42.330637 25528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:42.317773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48468 (local address 127.20.231.190:32865)
0619 16:58:42.318456 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:42.318467 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:42.318501 (+ 34us) server_negotiation.cc:407] Connection header received
0619 16:58:42.318646 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:42.318652 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:42.318719 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:42.319632 (+ 913us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:42.319732 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:42.321335 (+ 1603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:42.322389 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:42.322940 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:42.323040 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:42.325930 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:42.325935 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:42.325935 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:42.325948 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:42.328691 (+ 2743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:42.329132 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:42.329135 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:42.329136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:42.329250 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:42.329495 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:42.329495 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:42.329496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:42.329854 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:42.330017 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:42.330131 (+ 114us) server_negotiation.cc:299] Negotiation successful
0619 16:58:42.330399 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":565,"threads_started":1}
I20260619 16:58:42.333849 25533 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:58:42.334430 25533 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:58:42.338398 25533 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa1cccb46bb8464393cddc057ada1e6b: Generated new TSK 0
I20260619 16:58:42.338601 25533 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.231.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab.
Entry for principal kudu/127.20.231.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab.
I20260619 16:58:42.369951 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.129:0
--local_ip_for_outbound_sockets=127.20.231.129
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:32865
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:35023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:42.472082 25550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:42.472306 25550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:42.472332 25550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:42.473546 25550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:42.473596 25550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:42.473625 25550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.129
I20260619 16:58:42.475253 25550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:35023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:32865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.129
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:42.475534 25550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:42.475800 25550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:42.480921 25555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:42.480921 25556 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:42.481025 25558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:42.481352 25550 server_base.cc:1034] running on GCE node
I20260619 16:58:42.481599 25550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:42.481876 25550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:42.483036 25550 hybrid_clock.cc:648] HybridClock initialized: now 1781888322482996 us; error 78 us; skew 500 ppm
Jun 19 16:58:42 dist-test-slave-vnbj krb5kdc[25444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888322, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:42.485675 25550 init.cc:377] Logged in from keytab as kudu/127.20.231.129@KRBTEST.COM (short username kudu)
I20260619 16:58:42.486140 25550 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:42.486210 25550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:42.486352 25550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:42.487564 25550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-0/data/instance:
uuid: "bd78fc26cfbc4fa9b791795af73cf8a2"
format_stamp: "Formatted at 2026-06-19 16:58:42 on dist-test-slave-vnbj"
server_key: "62e69782734ecc82a56c2e8ecbadf804"
server_key_iv: "d325aca3b2363ce87c53004a26101a6b"
server_key_version: "encryptionkey@0"
I20260619 16:58:42.487993 25550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance:
uuid: "bd78fc26cfbc4fa9b791795af73cf8a2"
format_stamp: "Formatted at 2026-06-19 16:58:42 on dist-test-slave-vnbj"
server_key: "62e69782734ecc82a56c2e8ecbadf804"
server_key_iv: "d325aca3b2363ce87c53004a26101a6b"
server_key_version: "encryptionkey@0"
I20260619 16:58:42.489423 25550 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:58:42.490235 25564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:42.490454 25550 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:42.490553 25550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-0/wal
uuid: "bd78fc26cfbc4fa9b791795af73cf8a2"
format_stamp: "Formatted at 2026-06-19 16:58:42 on dist-test-slave-vnbj"
server_key: "62e69782734ecc82a56c2e8ecbadf804"
server_key_iv: "d325aca3b2363ce87c53004a26101a6b"
server_key_version: "encryptionkey@0"
I20260619 16:58:42.490641 25550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:42.502738 25550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:42.504838 25550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:42.505012 25550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:42.505251 25550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:58:42.505579 25550 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:42.505636 25550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:42.505683 25550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:42.505707 25550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:42.511498 25550 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.129:40561
I20260619 16:58:42.511579 25677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.129:40561 every 8 connection(s)
I20260619 16:58:42.511866 25550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
I20260619 16:58:42.516038 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 25550
I20260619 16:58:42.516144 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance
I20260619 16:58:42.516379 21406 external_mini_cluster.cc:1428] Setting key 48ccbda85964e6a88f4604a4e187d22e
Jun 19 16:58:42 dist-test-slave-vnbj krb5kdc[25444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888322, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:42.521905 25528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:42.512629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:54907 (local address 127.20.231.190:32865)
0619 16:58:42.512719 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:42.512721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:42.512937 (+ 216us) server_negotiation.cc:407] Connection header received
0619 16:58:42.513399 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:42.513400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:42.513414 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:42.513463 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:42.514392 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:42.514948 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:42.515995 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:42.516270 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:42.516329 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:42.518831 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:42.518833 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:42.518834 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:42.518880 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:42.520365 (+ 1485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:42.520918 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:42.520919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:42.520920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:42.520973 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:42.521255 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:42.521255 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:42.521255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:42.521367 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:42.521420 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:42.521769 (+ 349us) server_negotiation.cc:299] Negotiation successful
0619 16:58:42.521838 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 16:58:42.522245 25680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:42.512585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:32865 (local address 127.20.231.129:54907)
0619 16:58:42.512804 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:42.512840 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:42.513308 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:42.513523 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:42.513526 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:42.513796 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:42.514302 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:42.514306 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:42.515040 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:42.515041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:42.515905 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:42.515908 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:42.516355 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:42.516356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:42.516507 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:42.517117 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:42.517125 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:42.518729 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:42.520475 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:42.520483 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:42.520485 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:42.520844 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:42.521040 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:42.521041 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:42.521042 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:42.521188 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:42.521434 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:42.521440 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:42.521621 (+ 181us) client_negotiation.cc:769] Sending connection context
0619 16:58:42.521753 (+ 132us) client_negotiation.cc:240] Negotiation successful
0619 16:58:42.521939 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":91,"threads_started":1}
I20260619 16:58:42.522819 25678 heartbeater.cc:347] Connected to a master server at 127.20.231.190:32865
I20260619 16:58:42.522945 25678 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:42.523113 25678 heartbeater.cc:511] Master 127.20.231.190:32865 requested a full tablet report, sending...
I20260619 16:58:42.523622 25488 ts_manager.cc:194] Registered new tserver with Master: bd78fc26cfbc4fa9b791795af73cf8a2 (127.20.231.129:40561)
I20260619 16:58:42.524228 25488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.129@KRBTEST.COM'} at 127.20.231.129:54907
WARNING: no policy specified for kudu/127.20.231.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab.
Entry for principal kudu/127.20.231.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab.
I20260619 16:58:42.542407 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.130:0
--local_ip_for_outbound_sockets=127.20.231.130
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:32865
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:35023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:42.624241 25683 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:42.624454 25683 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:42.624477 25683 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:42.625674 25683 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:42.625718 25683 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:42.625746 25683 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.130
I20260619 16:58:42.627282 25683 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:35023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:32865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25683
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.130
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:42.627532 25683 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:42.627759 25683 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:42.632929 25688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:42.632929 25691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:42.632938 25689 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:42.633131 25683 server_base.cc:1034] running on GCE node
I20260619 16:58:42.633442 25683 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:42.633711 25683 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:42.634910 25683 hybrid_clock.cc:648] HybridClock initialized: now 1781888322634885 us; error 40 us; skew 500 ppm
Jun 19 16:58:42 dist-test-slave-vnbj krb5kdc[25444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888322, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:42.637494 25683 init.cc:377] Logged in from keytab as kudu/127.20.231.130@KRBTEST.COM (short username kudu)
I20260619 16:58:42.637913 25683 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:42.637986 25683 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:42.638127 25683 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:42.639338 25683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-1/data/instance:
uuid: "432f02ce038047eaae32b6963e128f0c"
format_stamp: "Formatted at 2026-06-19 16:58:42 on dist-test-slave-vnbj"
server_key: "e1a9fbed6bcac666b6e68d58c8f6623c"
server_key_iv: "40157399b73b3236062b0ea9accec5dc"
server_key_version: "encryptionkey@0"
I20260619 16:58:42.639704 25683 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance:
uuid: "432f02ce038047eaae32b6963e128f0c"
format_stamp: "Formatted at 2026-06-19 16:58:42 on dist-test-slave-vnbj"
server_key: "e1a9fbed6bcac666b6e68d58c8f6623c"
server_key_iv: "40157399b73b3236062b0ea9accec5dc"
server_key_version: "encryptionkey@0"
I20260619 16:58:42.641045 25683 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:58:42.641813 25697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:42.641996 25683 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:42.642096 25683 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-1/wal
uuid: "432f02ce038047eaae32b6963e128f0c"
format_stamp: "Formatted at 2026-06-19 16:58:42 on dist-test-slave-vnbj"
server_key: "e1a9fbed6bcac666b6e68d58c8f6623c"
server_key_iv: "40157399b73b3236062b0ea9accec5dc"
server_key_version: "encryptionkey@0"
I20260619 16:58:42.642213 25683 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:42.655740 25683 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:42.657706 25683 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:42.657825 25683 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:42.658012 25683 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:58:42.658344 25683 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:42.658380 25683 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:42.658403 25683 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:42.658452 25683 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:42.664495 25683 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.130:37479
I20260619 16:58:42.664534 25810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.130:37479 every 8 connection(s)
I20260619 16:58:42.664831 25683 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
I20260619 16:58:42.667609 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 25683
I20260619 16:58:42.667718 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance
I20260619 16:58:42.667892 21406 external_mini_cluster.cc:1428] Setting key cb83d1c741e0ec4c9ccca772e2dc4816
Jun 19 16:58:42 dist-test-slave-vnbj krb5kdc[25444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888322, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:42.676115 25528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:42.665709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:44599 (local address 127.20.231.190:32865)
0619 16:58:42.665768 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:42.665770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:42.665992 (+ 222us) server_negotiation.cc:407] Connection header received
0619 16:58:42.666455 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:42.666457 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:42.666468 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:42.666525 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:42.667614 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:42.668366 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:42.670072 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:42.670390 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:42.670469 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:42.672893 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:42.672894 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:42.672894 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:42.672901 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:42.674410 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:42.674970 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:42.674970 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:42.674971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:42.675026 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:42.675352 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:42.675353 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:42.675354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:42.675476 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:42.675587 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:42.675965 (+ 378us) server_negotiation.cc:299] Negotiation successful
0619 16:58:42.676039 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
WARNING: no policy specified for kudu/127.20.231.131@KRBTEST.COM; defaulting to no policy
I20260619 16:58:42.676342 25813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:42.665659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:32865 (local address 127.20.231.130:44599)
0619 16:58:42.665860 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:42.665898 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:42.666366 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:42.666605 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:42.666608 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:42.666981 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:42.667498 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:42.667504 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:42.668474 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:42.668475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:42.669977 (+ 1502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:42.669980 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:42.670483 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:42.670484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:42.670590 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:42.671234 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:42.671249 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:42.672786 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:42.674499 (+ 1713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:42.674507 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:42.674510 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:42.674891 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:42.675096 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:42.675097 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:42.675098 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:42.675240 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:42.675591 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:42.675597 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:42.675794 (+ 197us) client_negotiation.cc:769] Sending connection context
0619 16:58:42.675923 (+ 129us) client_negotiation.cc:240] Negotiation successful
0619 16:58:42.676103 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":71,"threads_started":1}
I20260619 16:58:42.676858 25811 heartbeater.cc:347] Connected to a master server at 127.20.231.190:32865
I20260619 16:58:42.676972 25811 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:42.677125 25811 heartbeater.cc:511] Master 127.20.231.190:32865 requested a full tablet report, sending...
I20260619 16:58:42.677594 25488 ts_manager.cc:194] Registered new tserver with Master: 432f02ce038047eaae32b6963e128f0c (127.20.231.130:37479)
I20260619 16:58:42.677942 25488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.130@KRBTEST.COM'} at 127.20.231.130:44599
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab.
Entry for principal kudu/127.20.231.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab.
I20260619 16:58:42.694264 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.131:0
--local_ip_for_outbound_sockets=127.20.231.131
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:32865
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:35023
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:42.777680 25816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:42.777904 25816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:42.777928 25816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:42.779285 25816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:42.779331 25816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:42.779359 25816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.131
I20260619 16:58:42.780822 25816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:35023
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:32865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.131
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:42.781004 25816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:42.781179 25816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:42.786662 25824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:42.786863 25821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:42.786940 25822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:42.787025 25816 server_base.cc:1034] running on GCE node
I20260619 16:58:42.787354 25816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:42.787628 25816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:42.788782 25816 hybrid_clock.cc:648] HybridClock initialized: now 1781888322788752 us; error 49 us; skew 500 ppm
Jun 19 16:58:42 dist-test-slave-vnbj krb5kdc[25444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888322, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:42.791625 25816 init.cc:377] Logged in from keytab as kudu/127.20.231.131@KRBTEST.COM (short username kudu)
I20260619 16:58:42.792068 25816 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:42.792119 25816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:42.792260 25816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:42.793421 25816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-2/data/instance:
uuid: "69e86875e62e471ea118a1e5ef334c4d"
format_stamp: "Formatted at 2026-06-19 16:58:42 on dist-test-slave-vnbj"
server_key: "83fb8c7e2c05bb61d5753d572347c01d"
server_key_iv: "99a2c6ce35b844c7cbaf358a480c99b9"
server_key_version: "encryptionkey@0"
I20260619 16:58:42.793862 25816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance:
uuid: "69e86875e62e471ea118a1e5ef334c4d"
format_stamp: "Formatted at 2026-06-19 16:58:42 on dist-test-slave-vnbj"
server_key: "83fb8c7e2c05bb61d5753d572347c01d"
server_key_iv: "99a2c6ce35b844c7cbaf358a480c99b9"
server_key_version: "encryptionkey@0"
I20260619 16:58:42.795418 25816 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:58:42.796288 25830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:42.796478 25816 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:58:42.796582 25816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-2/wal
uuid: "69e86875e62e471ea118a1e5ef334c4d"
format_stamp: "Formatted at 2026-06-19 16:58:42 on dist-test-slave-vnbj"
server_key: "83fb8c7e2c05bb61d5753d572347c01d"
server_key_iv: "99a2c6ce35b844c7cbaf358a480c99b9"
server_key_version: "encryptionkey@0"
I20260619 16:58:42.796658 25816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:42.814462 25816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:42.816736 25816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:42.816901 25816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:42.817147 25816 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:58:42.817471 25816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:42.817525 25816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:42.817569 25816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:42.817593 25816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:42.824126 25816 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.131:44377
I20260619 16:58:42.824157 25943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.131:44377 every 8 connection(s)
I20260619 16:58:42.824450 25816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
I20260619 16:58:42.829371 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 25816
I20260619 16:58:42.829453 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance
I20260619 16:58:42.829681 21406 external_mini_cluster.cc:1428] Setting key a9d1a654062f914bff5f177d096dea37
Jun 19 16:58:42 dist-test-slave-vnbj krb5kdc[25444](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888322, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:42.835152 25528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:42.825178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:49941 (local address 127.20.231.190:32865)
0619 16:58:42.825239 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:42.825241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:42.825453 (+ 212us) server_negotiation.cc:407] Connection header received
0619 16:58:42.825938 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:42.825939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:42.825950 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:42.826003 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:42.827223 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:42.827715 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:42.828842 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:42.829041 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:42.829097 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:42.831784 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:42.831785 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:42.831785 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:42.831792 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:42.833320 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:42.833893 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:42.833893 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:42.833894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:42.833963 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:42.834280 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:42.834281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:42.834282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:42.834414 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:42.834487 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:42.834978 (+ 491us) server_negotiation.cc:299] Negotiation successful
0619 16:58:42.835071 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 16:58:42.835304 25946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:42.825165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:32865 (local address 127.20.231.131:49941)
0619 16:58:42.825328 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:42.825365 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:42.825850 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:42.826079 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:42.826083 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:42.826389 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:42.827108 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:42.827115 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:42.827829 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:42.827830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:42.828756 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:42.828758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:42.829197 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:42.829199 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:42.829354 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:42.830021 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:42.830030 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:42.831699 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:42.833448 (+ 1749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:42.833455 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:42.833457 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:42.833769 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:42.834059 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:42.834060 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:42.834061 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:42.834178 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:42.834506 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:42.834511 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:42.834648 (+ 137us) client_negotiation.cc:769] Sending connection context
0619 16:58:42.834819 (+ 171us) client_negotiation.cc:240] Negotiation successful
0619 16:58:42.835043 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":94,"threads_started":1}
I20260619 16:58:42.835863 25944 heartbeater.cc:347] Connected to a master server at 127.20.231.190:32865
I20260619 16:58:42.835952 25944 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:42.836068 25944 heartbeater.cc:511] Master 127.20.231.190:32865 requested a full tablet report, sending...
I20260619 16:58:42.836406 25488 ts_manager.cc:194] Registered new tserver with Master: 69e86875e62e471ea118a1e5ef334c4d (127.20.231.131:44377)
I20260619 16:58:42.836858 25488 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.131@KRBTEST.COM'} at 127.20.231.131:49941
I20260619 16:58:42.841535 21406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 16:58:42.848898 25528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:42.842905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48474 (local address 127.20.231.190:32865)
0619 16:58:42.842968 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:42.842970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:42.843032 (+ 62us) server_negotiation.cc:407] Connection header received
0619 16:58:42.843102 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:42.843103 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:42.843111 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:42.843189 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:42.843859 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:42.844354 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:42.845183 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:42.845365 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:42.845424 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:42.846618 (+ 1194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:42.846619 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:42.846620 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:42.846626 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:42.847952 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:42.848263 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:42.848263 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:42.848264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:42.848313 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:42.848478 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:42.848479 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:42.848479 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:42.848614 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:42.848686 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:42.848776 (+ 90us) server_negotiation.cc:299] Negotiation successful
0619 16:58:42.848836 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 16:58:42.850641 25488 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48474:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:58:42.852247 25488 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:58:42.859203 25957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:42.854988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:40561 (local address 127.0.0.1:56426)
0619 16:58:42.855144 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:42.855157 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:42.855201 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:42.855806 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:42.855807 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:42.855810 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:42.855855 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:42.855856 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:42.857111 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:42.857112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:42.858181 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:42.858183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:42.858881 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:42.858882 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:42.858965 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:42.858977 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 16:58:42.859052 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 16:58:42.859105 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":100,"threads_started":1}
I20260619 16:58:42.859318 25958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:42.855210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:37479 (local address 127.0.0.1:54378)
0619 16:58:42.855430 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:42.855441 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:42.855493 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:42.855817 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:42.855818 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:42.855821 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:42.855873 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:42.855874 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:42.856838 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:42.856839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:42.858043 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:42.858046 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:42.859073 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:42.859073 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:42.859123 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:42.859127 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:58:42.859169 (+ 42us) client_negotiation.cc:240] Negotiation successful
0619 16:58:42.859201 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":30,"thread_start_us":71,"threads_started":1}
I20260619 16:58:42.859484 25959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:42.855333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54378 (local address 127.20.231.130:37479)
0619 16:58:42.855660 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:42.855665 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:42.855675 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:58:42.855718 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:42.855720 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:42.855737 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:42.855821 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:42.855947 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:42.856719 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:42.858142 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:42.858978 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:42.859077 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:42.859160 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:42.859191 (+ 31us) server_negotiation.cc:299] Negotiation successful
0619 16:58:42.859234 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":87,"threads_started":1}
I20260619 16:58:42.859529 25955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:42.854739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:44377 (local address 127.0.0.1:42448)
0619 16:58:42.854882 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:42.854926 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:42.854989 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:42.855316 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:42.855319 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:42.855321 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:42.855481 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:42.855484 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:42.856830 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:42.856831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:42.858130 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:42.858132 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:42.859216 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:42.859217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:42.859309 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:42.859323 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 16:58:42.859384 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 16:58:42.859442 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":37,"thread_start_us":73,"threads_started":1}
I20260619 16:58:42.859777 25956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:42.854823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42448 (local address 127.20.231.131:44377)
0619 16:58:42.855036 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:42.855041 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:42.855054 (+ 13us) server_negotiation.cc:407] Connection header received
0619 16:58:42.855098 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:42.855101 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:42.855119 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:42.855214 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:42.855574 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:42.856721 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:42.858246 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:42.859058 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:42.859127 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:42.859178 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:42.859645 (+ 467us) server_negotiation.cc:299] Negotiation successful
0619 16:58:42.859693 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":147,"threads_started":1}
I20260619 16:58:42.859925 25960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:42.855448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56426 (local address 127.20.231.129:40561)
0619 16:58:42.855642 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:42.855646 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:42.855655 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:58:42.855691 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:42.855693 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:42.855707 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:42.855921 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:42.855997 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:42.857004 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:42.858288 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:42.858769 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:42.858907 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:42.858977 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:42.859773 (+ 796us) server_negotiation.cc:299] Negotiation successful
0619 16:58:42.859825 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":84,"threads_started":1}
I20260619 16:58:42.860263 25745 tablet_service.cc:1467] Processing CreateTablet for tablet 92569a88ccff49bd954875f2bd28b33a (DEFAULT_TABLE table=test-table [id=6645832df9584f70ae1b04bb6919c285]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:42.860651 25745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92569a88ccff49bd954875f2bd28b33a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:42.860590 25612 tablet_service.cc:1467] Processing CreateTablet for tablet 92569a88ccff49bd954875f2bd28b33a (DEFAULT_TABLE table=test-table [id=6645832df9584f70ae1b04bb6919c285]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:42.860817 25878 tablet_service.cc:1467] Processing CreateTablet for tablet 92569a88ccff49bd954875f2bd28b33a (DEFAULT_TABLE table=test-table [id=6645832df9584f70ae1b04bb6919c285]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:42.860946 25612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92569a88ccff49bd954875f2bd28b33a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:42.861151 25878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92569a88ccff49bd954875f2bd28b33a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:42.863867 25961 tablet_bootstrap.cc:492] T 92569a88ccff49bd954875f2bd28b33a P 69e86875e62e471ea118a1e5ef334c4d: Bootstrap starting.
I20260619 16:58:42.864425 25962 tablet_bootstrap.cc:492] T 92569a88ccff49bd954875f2bd28b33a P 432f02ce038047eaae32b6963e128f0c: Bootstrap starting.
I20260619 16:58:42.864585 25963 tablet_bootstrap.cc:492] T 92569a88ccff49bd954875f2bd28b33a P bd78fc26cfbc4fa9b791795af73cf8a2: Bootstrap starting.
I20260619 16:58:42.864665 25961 tablet_bootstrap.cc:654] T 92569a88ccff49bd954875f2bd28b33a P 69e86875e62e471ea118a1e5ef334c4d: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:42.865110 25962 tablet_bootstrap.cc:654] T 92569a88ccff49bd954875f2bd28b33a P 432f02ce038047eaae32b6963e128f0c: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:42.865217 25963 tablet_bootstrap.cc:654] T 92569a88ccff49bd954875f2bd28b33a P bd78fc26cfbc4fa9b791795af73cf8a2: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:42.865290 25961 log.cc:824] T 92569a88ccff49bd954875f2bd28b33a P 69e86875e62e471ea118a1e5ef334c4d: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:42.865702 25962 log.cc:824] T 92569a88ccff49bd954875f2bd28b33a P 432f02ce038047eaae32b6963e128f0c: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:42.865875 25963 log.cc:824] T 92569a88ccff49bd954875f2bd28b33a P bd78fc26cfbc4fa9b791795af73cf8a2: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:42.866240 25961 tablet_bootstrap.cc:492] T 92569a88ccff49bd954875f2bd28b33a P 69e86875e62e471ea118a1e5ef334c4d: No bootstrap required, opened a new log
I20260619 16:58:42.866328 25961 ts_tablet_manager.cc:1397] T 92569a88ccff49bd954875f2bd28b33a P 69e86875e62e471ea118a1e5ef334c4d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 16:58:42.866600 25962 tablet_bootstrap.cc:492] T 92569a88ccff49bd954875f2bd28b33a P 432f02ce038047eaae32b6963e128f0c: No bootstrap required, opened a new log
I20260619 16:58:42.866668 25962 ts_tablet_manager.cc:1397] T 92569a88ccff49bd954875f2bd28b33a P 432f02ce038047eaae32b6963e128f0c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 16:58:42.867201 25963 tablet_bootstrap.cc:492] T 92569a88ccff49bd954875f2bd28b33a P bd78fc26cfbc4fa9b791795af73cf8a2: No bootstrap required, opened a new log
I20260619 16:58:42.867275 25963 ts_tablet_manager.cc:1397] T 92569a88ccff49bd954875f2bd28b33a P bd78fc26cfbc4fa9b791795af73cf8a2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 16:58:42.868628 25963 raft_consensus.cc:348] T 92569a88ccff49bd954875f2bd28b33a P bd78fc26cfbc4fa9b791795af73cf8a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e86875e62e471ea118a1e5ef334c4d" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 44377 } } peers { permanent_uuid: "bd78fc26cfbc4fa9b791795af73cf8a2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 40561 } } peers { permanent_uuid: "432f02ce038047eaae32b6963e128f0c" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 37479 } }
I20260619 16:58:42.868606 25961 raft_consensus.cc:348] T 92569a88ccff49bd954875f2bd28b33a P 69e86875e62e471ea118a1e5ef334c4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e86875e62e471ea118a1e5ef334c4d" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 44377 } } peers { permanent_uuid: "bd78fc26cfbc4fa9b791795af73cf8a2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 40561 } } peers { permanent_uuid: "432f02ce038047eaae32b6963e128f0c" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 37479 } }
I20260619 16:58:42.868798 25961 raft_consensus.cc:374] T 92569a88ccff49bd954875f2bd28b33a P 69e86875e62e471ea118a1e5ef334c4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:42.868799 25963 raft_consensus.cc:374] T 92569a88ccff49bd954875f2bd28b33a P bd78fc26cfbc4fa9b791795af73cf8a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:42.868851 25963 raft_consensus.cc:729] T 92569a88ccff49bd954875f2bd28b33a P bd78fc26cfbc4fa9b791795af73cf8a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd78fc26cfbc4fa9b791795af73cf8a2, State: Initialized, Role: FOLLOWER
I20260619 16:58:42.868852 25961 raft_consensus.cc:729] T 92569a88ccff49bd954875f2bd28b33a P 69e86875e62e471ea118a1e5ef334c4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69e86875e62e471ea118a1e5ef334c4d, State: Initialized, Role: FOLLOWER
I20260619 16:58:42.868829 25962 raft_consensus.cc:348] T 92569a88ccff49bd954875f2bd28b33a P 432f02ce038047eaae32b6963e128f0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e86875e62e471ea118a1e5ef334c4d" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 44377 } } peers { permanent_uuid: "bd78fc26cfbc4fa9b791795af73cf8a2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 40561 } } peers { permanent_uuid: "432f02ce038047eaae32b6963e128f0c" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 37479 } }
I20260619 16:58:42.868976 25962 raft_consensus.cc:374] T 92569a88ccff49bd954875f2bd28b33a P 432f02ce038047eaae32b6963e128f0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:42.868970 25961 consensus_queue.cc:260] T 92569a88ccff49bd954875f2bd28b33a P 69e86875e62e471ea118a1e5ef334c4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e86875e62e471ea118a1e5ef334c4d" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 44377 } } peers { permanent_uuid: "bd78fc26cfbc4fa9b791795af73cf8a2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 40561 } } peers { permanent_uuid: "432f02ce038047eaae32b6963e128f0c" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 37479 } }
I20260619 16:58:42.869019 25962 raft_consensus.cc:729] T 92569a88ccff49bd954875f2bd28b33a P 432f02ce038047eaae32b6963e128f0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 432f02ce038047eaae32b6963e128f0c, State: Initialized, Role: FOLLOWER
I20260619 16:58:42.868986 25963 consensus_queue.cc:260] T 92569a88ccff49bd954875f2bd28b33a P bd78fc26cfbc4fa9b791795af73cf8a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e86875e62e471ea118a1e5ef334c4d" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 44377 } } peers { permanent_uuid: "bd78fc26cfbc4fa9b791795af73cf8a2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 40561 } } peers { permanent_uuid: "432f02ce038047eaae32b6963e128f0c" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 37479 } }
I20260619 16:58:42.869144 25962 consensus_queue.cc:260] T 92569a88ccff49bd954875f2bd28b33a P 432f02ce038047eaae32b6963e128f0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e86875e62e471ea118a1e5ef334c4d" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 44377 } } peers { permanent_uuid: "bd78fc26cfbc4fa9b791795af73cf8a2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 40561 } } peers { permanent_uuid: "432f02ce038047eaae32b6963e128f0c" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 37479 } }
I20260619 16:58:42.869297 25963 ts_tablet_manager.cc:1428] T 92569a88ccff49bd954875f2bd28b33a P bd78fc26cfbc4fa9b791795af73cf8a2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 16:58:42.869305 25961 ts_tablet_manager.cc:1428] T 92569a88ccff49bd954875f2bd28b33a P 69e86875e62e471ea118a1e5ef334c4d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 16:58:42.869330 25678 heartbeater.cc:503] Master 127.20.231.190:32865 was elected leader, sending a full tablet report...
I20260619 16:58:42.869438 25962 ts_tablet_manager.cc:1428] T 92569a88ccff49bd954875f2bd28b33a P 432f02ce038047eaae32b6963e128f0c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 16:58:42.869451 25944 heartbeater.cc:503] Master 127.20.231.190:32865 was elected leader, sending a full tablet report...
I20260619 16:58:42.869575 25811 heartbeater.cc:503] Master 127.20.231.190:32865 was elected leader, sending a full tablet report...
W20260619 16:58:42.915401 25812 tablet.cc:2369] T 92569a88ccff49bd954875f2bd28b33a P 432f02ce038047eaae32b6963e128f0c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 16:58:43.012816 25679 tablet.cc:2369] T 92569a88ccff49bd954875f2bd28b33a P bd78fc26cfbc4fa9b791795af73cf8a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 16:58:43.075052 25945 tablet.cc:2369] T 92569a88ccff49bd954875f2bd28b33a P 69e86875e62e471ea118a1e5ef334c4d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:58:43.166054 25967 raft_consensus.cc:482] T 92569a88ccff49bd954875f2bd28b33a P bd78fc26cfbc4fa9b791795af73cf8a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:58:43.166260 25967 raft_consensus.cc:504] T 92569a88ccff49bd954875f2bd28b33a P bd78fc26cfbc4fa9b791795af73cf8a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e86875e62e471ea118a1e5ef334c4d" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 44377 } } peers { permanent_uuid: "bd78fc26cfbc4fa9b791795af73cf8a2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 40561 } } peers { permanent_uuid: "432f02ce038047eaae32b6963e128f0c" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 37479 } }
I20260619 16:58:43.166647 25967 leader_election.cc:288] T 92569a88ccff49bd954875f2bd28b33a P bd78fc26cfbc4fa9b791795af73cf8a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 69e86875e62e471ea118a1e5ef334c4d (127.20.231.131:44377), 432f02ce038047eaae32b6963e128f0c (127.20.231.130:37479)
I20260619 16:58:43.170024 25959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:43.167223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:39441 (local address 127.20.231.130:37479)
0619 16:58:43.167270 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:43.167272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:43.167353 (+ 81us) server_negotiation.cc:407] Connection header received
0619 16:58:43.167423 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:43.167424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:43.167434 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:43.167515 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:43.167712 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:43.168188 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:43.169262 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:43.169664 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:43.169752 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:43.169760 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:43.169920 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 16:58:43.169958 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 16:58:43.170022 25956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:43.167080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:53921 (local address 127.20.231.131:44377)
0619 16:58:43.167254 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:43.167256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:43.167351 (+ 95us) server_negotiation.cc:407] Connection header received
0619 16:58:43.167440 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:43.167441 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:43.167453 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:43.167522 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:43.167720 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:43.168225 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:43.169269 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:43.169656 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:43.169745 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:43.169754 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:43.169914 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 16:58:43.169949 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143}
I20260619 16:58:43.170045 25970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:43.166961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:44377 (local address 127.20.231.129:53921)
0619 16:58:43.167255 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:43.167268 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:43.167372 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:43.167527 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:43.167527 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:43.167529 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:43.167637 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:43.167639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:43.168310 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:43.168310 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:43.169159 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:43.169162 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:43.169743 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:43.169744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:43.169814 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:43.169829 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 16:58:43.169903 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 16:58:43.169947 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":36,"thread_start_us":78,"threads_started":1}
I20260619 16:58:43.170051 25971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:43.167153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:37479 (local address 127.20.231.129:39441)
0619 16:58:43.167288 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:43.167298 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:43.167364 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:43.167521 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:43.167521 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:43.167524 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:43.167607 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:43.167609 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:43.168282 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:43.168283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:43.169160 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:43.169162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:43.169752 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:43.169753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:43.169832 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:43.169835 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:58:43.169913 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 16:58:43.169957 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":32,"thread_start_us":70,"threads_started":1}
I20260619 16:58:43.170392 25765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92569a88ccff49bd954875f2bd28b33a" candidate_uuid: "bd78fc26cfbc4fa9b791795af73cf8a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "432f02ce038047eaae32b6963e128f0c" is_pre_election: true
I20260619 16:58:43.170549 25765 raft_consensus.cc:2454] T 92569a88ccff49bd954875f2bd28b33a P 432f02ce038047eaae32b6963e128f0c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd78fc26cfbc4fa9b791795af73cf8a2 in term 0.
I20260619 16:58:43.170614 25898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92569a88ccff49bd954875f2bd28b33a" candidate_uuid: "bd78fc26cfbc4fa9b791795af73cf8a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69e86875e62e471ea118a1e5ef334c4d" is_pre_election: true
I20260619 16:58:43.170809 25568 leader_election.cc:302] T 92569a88ccff49bd954875f2bd28b33a P bd78fc26cfbc4fa9b791795af73cf8a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 432f02ce038047eaae32b6963e128f0c, bd78fc26cfbc4fa9b791795af73cf8a2; no voters:
I20260619 16:58:43.170897 25898 raft_consensus.cc:2454] T 92569a88ccff49bd954875f2bd28b33a P 69e86875e62e471ea118a1e5ef334c4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd78fc26cfbc4fa9b791795af73cf8a2 in term 0.
I20260619 16:58:43.170979 25967 raft_consensus.cc:2781] T 92569a88ccff49bd954875f2bd28b33a P bd78fc26cfbc4fa9b791795af73cf8a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:58:43.171036 25967 raft_consensus.cc:482] T 92569a88ccff49bd954875f2bd28b33a P bd78fc26cfbc4fa9b791795af73cf8a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:58:43.171093 25967 raft_consensus.cc:3037] T 92569a88ccff49bd954875f2bd28b33a P bd78fc26cfbc4fa9b791795af73cf8a2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:43.171763 25967 raft_consensus.cc:504] T 92569a88ccff49bd954875f2bd28b33a P bd78fc26cfbc4fa9b791795af73cf8a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e86875e62e471ea118a1e5ef334c4d" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 44377 } } peers { permanent_uuid: "bd78fc26cfbc4fa9b791795af73cf8a2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 40561 } } peers { permanent_uuid: "432f02ce038047eaae32b6963e128f0c" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 37479 } }
I20260619 16:58:43.171869 25967 leader_election.cc:288] T 92569a88ccff49bd954875f2bd28b33a P bd78fc26cfbc4fa9b791795af73cf8a2 [CANDIDATE]: Term 1 election: Requested vote from peers 69e86875e62e471ea118a1e5ef334c4d (127.20.231.131:44377), 432f02ce038047eaae32b6963e128f0c (127.20.231.130:37479)
I20260619 16:58:43.172104 25898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92569a88ccff49bd954875f2bd28b33a" candidate_uuid: "bd78fc26cfbc4fa9b791795af73cf8a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "69e86875e62e471ea118a1e5ef334c4d"
I20260619 16:58:43.172194 25898 raft_consensus.cc:3037] T 92569a88ccff49bd954875f2bd28b33a P 69e86875e62e471ea118a1e5ef334c4d [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:43.172178 25765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92569a88ccff49bd954875f2bd28b33a" candidate_uuid: "bd78fc26cfbc4fa9b791795af73cf8a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "432f02ce038047eaae32b6963e128f0c"
I20260619 16:58:43.172310 25765 raft_consensus.cc:3037] T 92569a88ccff49bd954875f2bd28b33a P 432f02ce038047eaae32b6963e128f0c [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:43.172955 25898 raft_consensus.cc:2454] T 92569a88ccff49bd954875f2bd28b33a P 69e86875e62e471ea118a1e5ef334c4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd78fc26cfbc4fa9b791795af73cf8a2 in term 1.
I20260619 16:58:43.173144 25568 leader_election.cc:302] T 92569a88ccff49bd954875f2bd28b33a P bd78fc26cfbc4fa9b791795af73cf8a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 69e86875e62e471ea118a1e5ef334c4d, bd78fc26cfbc4fa9b791795af73cf8a2; no voters:
I20260619 16:58:43.173261 25967 raft_consensus.cc:2781] T 92569a88ccff49bd954875f2bd28b33a P bd78fc26cfbc4fa9b791795af73cf8a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:43.173384 25765 raft_consensus.cc:2454] T 92569a88ccff49bd954875f2bd28b33a P 432f02ce038047eaae32b6963e128f0c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd78fc26cfbc4fa9b791795af73cf8a2 in term 1.
I20260619 16:58:43.173463 25967 raft_consensus.cc:686] T 92569a88ccff49bd954875f2bd28b33a P bd78fc26cfbc4fa9b791795af73cf8a2 [term 1 LEADER]: Becoming Leader. State: Replica: bd78fc26cfbc4fa9b791795af73cf8a2, State: Running, Role: LEADER
I20260619 16:58:43.173578 25967 consensus_queue.cc:237] T 92569a88ccff49bd954875f2bd28b33a P bd78fc26cfbc4fa9b791795af73cf8a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e86875e62e471ea118a1e5ef334c4d" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 44377 } } peers { permanent_uuid: "bd78fc26cfbc4fa9b791795af73cf8a2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 40561 } } peers { permanent_uuid: "432f02ce038047eaae32b6963e128f0c" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 37479 } }
I20260619 16:58:43.174643 25488 catalog_manager.cc:5495] T 92569a88ccff49bd954875f2bd28b33a P bd78fc26cfbc4fa9b791795af73cf8a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd78fc26cfbc4fa9b791795af73cf8a2 (127.20.231.129). New cstate: current_term: 1 leader_uuid: "bd78fc26cfbc4fa9b791795af73cf8a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "69e86875e62e471ea118a1e5ef334c4d" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 44377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bd78fc26cfbc4fa9b791795af73cf8a2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 40561 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "432f02ce038047eaae32b6963e128f0c" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 37479 } health_report { overall_health: UNKNOWN } } }
I20260619 16:58:43.213110 25960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:43.210823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56436 (local address 127.20.231.129:40561)
0619 16:58:43.210889 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:43.210896 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:43.210906 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:58:43.210945 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:43.210946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:43.210963 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:43.211036 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:58:43.211145 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:43.211611 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:43.212379 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:43.212576 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:43.212706 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:43.212830 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:58:43.212931 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:58:43.212990 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:43.213033 (+ 43us) server_negotiation.cc:299] Negotiation successful
0619 16:58:43.213052 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 16:58:43.214932 25765 raft_consensus.cc:1261] T 92569a88ccff49bd954875f2bd28b33a P 432f02ce038047eaae32b6963e128f0c [term 1 FOLLOWER]: Refusing update from remote peer bd78fc26cfbc4fa9b791795af73cf8a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:43.215129 25898 raft_consensus.cc:1261] T 92569a88ccff49bd954875f2bd28b33a P 69e86875e62e471ea118a1e5ef334c4d [term 1 FOLLOWER]: Refusing update from remote peer bd78fc26cfbc4fa9b791795af73cf8a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:43.215351 25967 consensus_queue.cc:1035] T 92569a88ccff49bd954875f2bd28b33a P bd78fc26cfbc4fa9b791795af73cf8a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "432f02ce038047eaae32b6963e128f0c" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 37479 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:43.215518 25972 consensus_queue.cc:1035] T 92569a88ccff49bd954875f2bd28b33a P bd78fc26cfbc4fa9b791795af73cf8a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "69e86875e62e471ea118a1e5ef334c4d" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 44377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:43.218290 25976 mvcc.cc:205] Tried to move back new op lower bound from 7298614571885670400 to 7298614571719929856. Current Snapshot: MvccSnapshot[applied={T|T < 7298614571885670400}]
I20260619 16:58:43.221345 25487 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48474:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:58:43.221472 25487 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48474:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:58:43.222289 25487 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fa1cccb46bb8464393cddc057ada1e6b: Sending DeleteTablet for 3 replicas of tablet 92569a88ccff49bd954875f2bd28b33a
I20260619 16:58:43.222627 25878 tablet_service.cc:1514] Processing DeleteTablet for tablet 92569a88ccff49bd954875f2bd28b33a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:43 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:42448
I20260619 16:58:43.222666 25612 tablet_service.cc:1514] Processing DeleteTablet for tablet 92569a88ccff49bd954875f2bd28b33a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:43 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:56426
I20260619 16:58:43.222918 25745 tablet_service.cc:1514] Processing DeleteTablet for tablet 92569a88ccff49bd954875f2bd28b33a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:43 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:54378
I20260619 16:58:43.223095 25985 tablet_replica.cc:330] T 92569a88ccff49bd954875f2bd28b33a P 69e86875e62e471ea118a1e5ef334c4d: stopping tablet replica
I20260619 16:58:43.223196 25985 raft_consensus.cc:2229] T 92569a88ccff49bd954875f2bd28b33a P 69e86875e62e471ea118a1e5ef334c4d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:58:43.223199 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 25550
I20260619 16:58:43.223037 25986 tablet_replica.cc:330] T 92569a88ccff49bd954875f2bd28b33a P bd78fc26cfbc4fa9b791795af73cf8a2: stopping tablet replica
I20260619 16:58:43.223814 25987 tablet_replica.cc:330] T 92569a88ccff49bd954875f2bd28b33a P 432f02ce038047eaae32b6963e128f0c: stopping tablet replica
I20260619 16:58:43.223878 25985 raft_consensus.cc:2258] T 92569a88ccff49bd954875f2bd28b33a P 69e86875e62e471ea118a1e5ef334c4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:43.223904 25987 raft_consensus.cc:2229] T 92569a88ccff49bd954875f2bd28b33a P 432f02ce038047eaae32b6963e128f0c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:58:43.223994 25987 raft_consensus.cc:2258] T 92569a88ccff49bd954875f2bd28b33a P 432f02ce038047eaae32b6963e128f0c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:43.224359 25985 ts_tablet_manager.cc:1905] T 92569a88ccff49bd954875f2bd28b33a P 69e86875e62e471ea118a1e5ef334c4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:43.225531 25987 ts_tablet_manager.cc:1905] T 92569a88ccff49bd954875f2bd28b33a P 432f02ce038047eaae32b6963e128f0c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:43.226222 25985 ts_tablet_manager.cc:1918] T 92569a88ccff49bd954875f2bd28b33a P 69e86875e62e471ea118a1e5ef334c4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:58:43.226310 25985 log.cc:1196] T 92569a88ccff49bd954875f2bd28b33a P 69e86875e62e471ea118a1e5ef334c4d: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-2/wal/wals/92569a88ccff49bd954875f2bd28b33a
I20260619 16:58:43.226589 25985 ts_tablet_manager.cc:1939] T 92569a88ccff49bd954875f2bd28b33a P 69e86875e62e471ea118a1e5ef334c4d: Deleting consensus metadata
I20260619 16:58:43.227027 25475 catalog_manager.cc:4841] TS 69e86875e62e471ea118a1e5ef334c4d (127.20.231.131:44377): tablet 92569a88ccff49bd954875f2bd28b33a (table test-table [id=6645832df9584f70ae1b04bb6919c285]) successfully deleted
I20260619 16:58:43.227248 25987 ts_tablet_manager.cc:1918] T 92569a88ccff49bd954875f2bd28b33a P 432f02ce038047eaae32b6963e128f0c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:58:43.227306 25987 log.cc:1196] T 92569a88ccff49bd954875f2bd28b33a P 432f02ce038047eaae32b6963e128f0c: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781888301345144-21406-0/minicluster-data/ts-1/wal/wals/92569a88ccff49bd954875f2bd28b33a
I20260619 16:58:43.227555 25987 ts_tablet_manager.cc:1939] T 92569a88ccff49bd954875f2bd28b33a P 432f02ce038047eaae32b6963e128f0c: Deleting consensus metadata
I20260619 16:58:43.227972 25475 catalog_manager.cc:4841] TS 432f02ce038047eaae32b6963e128f0c (127.20.231.130:37479): tablet 92569a88ccff49bd954875f2bd28b33a (table test-table [id=6645832df9584f70ae1b04bb6919c285]) successfully deleted
W20260619 16:58:43.229326 25475 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.231.129:40561 (error 108)
I20260619 16:58:43.229586 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 25683
I20260619 16:58:43.230242 25955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:43.229754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:40561 (local address 127.0.0.1:56448)
0619 16:58:43.230160 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:43.230202 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.129:40561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380}
W20260619 16:58:43.230382 25475 catalog_manager.cc:4584] TS bd78fc26cfbc4fa9b791795af73cf8a2 (127.20.231.129:40561): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 92569a88ccff49bd954875f2bd28b33a: Network error: Client connection negotiation failed: client connection to 127.20.231.129:40561: connect: Connection refused (error 111)
I20260619 16:58:43.234364 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 25816
I20260619 16:58:43.239048 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 25457
2026-06-19T16:58:43Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2239 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-06-19T16:58:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:58:43Z Disabled control of system clock
I20260619 16:58:43.266412 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:33769
--webserver_interface=127.20.231.190
--webserver_port=0
--builtin_ntp_servers=127.20.231.148:40761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:33769
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 16:58:43.350936 25992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:43.351143 25992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:43.351164 25992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:43.352339 25992 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:43.352382 25992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:43.352396 25992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:43.352409 25992 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:43.352421 25992 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:58:43.353782 25992 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:40761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:33769
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:33769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:43.354043 25992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:43.354353 25992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:43.359337 25998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:43.359344 25997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:43.359347 25992 server_base.cc:1034] running on GCE node
W20260619 16:58:43.359391 26000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:43.359797 25992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:43.360157 25992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:43.361348 25992 hybrid_clock.cc:648] HybridClock initialized: now 1781888323361347 us; error 64 us; skew 500 ppm
I20260619 16:58:43.362771 25992 webserver.cc:460] Webserver started at http://127.20.231.190:37841/ using document root <none> and password file <none>
I20260619 16:58:43.363049 25992 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:43.363121 25992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:43.363318 25992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:43.364488 25992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "04fcc4c6a92f469da5328d3f1449aa7c"
format_stamp: "Formatted at 2026-06-19 16:58:43 on dist-test-slave-vnbj"
server_key: "8c0b8929fc233db07e01bb11b31c81ad"
server_key_iv: "f4e6e5542d28647a36b24b7c816dd2a8"
server_key_version: "encryptionkey@0"
I20260619 16:58:43.364871 25992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "04fcc4c6a92f469da5328d3f1449aa7c"
format_stamp: "Formatted at 2026-06-19 16:58:43 on dist-test-slave-vnbj"
server_key: "8c0b8929fc233db07e01bb11b31c81ad"
server_key_iv: "f4e6e5542d28647a36b24b7c816dd2a8"
server_key_version: "encryptionkey@0"
I20260619 16:58:43.366276 25992 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:58:43.367225 26006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:43.367426 25992 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:43.367529 25992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "04fcc4c6a92f469da5328d3f1449aa7c"
format_stamp: "Formatted at 2026-06-19 16:58:43 on dist-test-slave-vnbj"
server_key: "8c0b8929fc233db07e01bb11b31c81ad"
server_key_iv: "f4e6e5542d28647a36b24b7c816dd2a8"
server_key_version: "encryptionkey@0"
I20260619 16:58:43.367612 25992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:43.371213 25992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:43.371490 25992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:43.371632 25992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:43.375411 25992 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:33769
I20260619 16:58:43.375474 26058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:33769 every 8 connection(s)
I20260619 16:58:43.375865 25992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 16:58:43.376534 26059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:43.379485 26059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04fcc4c6a92f469da5328d3f1449aa7c: Bootstrap starting.
I20260619 16:58:43.380074 26059 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04fcc4c6a92f469da5328d3f1449aa7c: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:43.380363 26059 log.cc:824] T 00000000000000000000000000000000 P 04fcc4c6a92f469da5328d3f1449aa7c: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:43.381072 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 25992
I20260619 16:58:43.381170 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 16:58:43.381194 26059 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04fcc4c6a92f469da5328d3f1449aa7c: No bootstrap required, opened a new log
I20260619 16:58:43.381374 21406 external_mini_cluster.cc:1428] Setting key a621a303d609179a542b913b9936ab87
I20260619 16:58:43.383677 26059 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04fcc4c6a92f469da5328d3f1449aa7c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04fcc4c6a92f469da5328d3f1449aa7c" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 33769 } }
I20260619 16:58:43.383875 26059 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04fcc4c6a92f469da5328d3f1449aa7c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:43.383926 26059 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04fcc4c6a92f469da5328d3f1449aa7c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04fcc4c6a92f469da5328d3f1449aa7c, State: Initialized, Role: FOLLOWER
I20260619 16:58:43.384007 26063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:43.381908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49230 (local address 127.20.231.190:33769)
0619 16:58:43.382108 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:43.382115 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:43.382140 (+ 25us) server_negotiation.cc:407] Connection header received
0619 16:58:43.382570 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:43.382576 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:43.382613 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:43.382732 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:43.383475 (+ 743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:43.383478 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:43.383478 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:58:43.383482 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:43.383525 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:58:43.383535 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:43.383590 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:43.383685 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 16:58:43.383763 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":98,"threads_started":1}
I20260619 16:58:43.384054 26059 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04fcc4c6a92f469da5328d3f1449aa7c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04fcc4c6a92f469da5328d3f1449aa7c" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 33769 } }
I20260619 16:58:43.384160 26059 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04fcc4c6a92f469da5328d3f1449aa7c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:58:43.384198 26059 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04fcc4c6a92f469da5328d3f1449aa7c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:58:43.384246 26059 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04fcc4c6a92f469da5328d3f1449aa7c [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:43.385079 26059 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04fcc4c6a92f469da5328d3f1449aa7c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04fcc4c6a92f469da5328d3f1449aa7c" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 33769 } }
I20260619 16:58:43.385198 26059 leader_election.cc:302] T 00000000000000000000000000000000 P 04fcc4c6a92f469da5328d3f1449aa7c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04fcc4c6a92f469da5328d3f1449aa7c; no voters:
I20260619 16:58:43.385380 26059 leader_election.cc:288] T 00000000000000000000000000000000 P 04fcc4c6a92f469da5328d3f1449aa7c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:58:43.385409 26064 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04fcc4c6a92f469da5328d3f1449aa7c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:43.385565 26064 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04fcc4c6a92f469da5328d3f1449aa7c [term 1 LEADER]: Becoming Leader. State: Replica: 04fcc4c6a92f469da5328d3f1449aa7c, State: Running, Role: LEADER
I20260619 16:58:43.385643 26059 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04fcc4c6a92f469da5328d3f1449aa7c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:43.385762 26064 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04fcc4c6a92f469da5328d3f1449aa7c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04fcc4c6a92f469da5328d3f1449aa7c" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 33769 } }
I20260619 16:58:43.386153 26065 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04fcc4c6a92f469da5328d3f1449aa7c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04fcc4c6a92f469da5328d3f1449aa7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04fcc4c6a92f469da5328d3f1449aa7c" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 33769 } } }
I20260619 16:58:43.386247 26065 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04fcc4c6a92f469da5328d3f1449aa7c [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:43.386402 26066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04fcc4c6a92f469da5328d3f1449aa7c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04fcc4c6a92f469da5328d3f1449aa7c. Latest consensus state: current_term: 1 leader_uuid: "04fcc4c6a92f469da5328d3f1449aa7c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04fcc4c6a92f469da5328d3f1449aa7c" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 33769 } } }
I20260619 16:58:43.386474 26066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04fcc4c6a92f469da5328d3f1449aa7c [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:43.386528 26073 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:58:43.387233 26073 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:58:43.388751 26073 catalog_manager.cc:1347] Generated new cluster ID: 98c5adafd6f44d2aa5e03dfe3b5c6e40
I20260619 16:58:43.388797 26073 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:58:43.400314 26073 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:58:43.400506 26073 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:58:43.412060 26073 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04fcc4c6a92f469da5328d3f1449aa7c: Generated new TSK 0
I20260619 16:58:43.412284 26073 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:58:43.414516 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.129:0
--local_ip_for_outbound_sockets=127.20.231.129
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:33769
--builtin_ntp_servers=127.20.231.148:40761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 16:58:43.506585 26083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:43.506844 26083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:43.506897 26083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:43.508143 26083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:43.508215 26083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:43.508261 26083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.129
I20260619 16:58:43.509764 26083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:40761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:33769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.129
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:43.510039 26083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:43.510347 26083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:43.516326 26089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:43.516326 26088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:43.516343 26091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:43.517040 26083 server_base.cc:1034] running on GCE node
I20260619 16:58:43.517263 26083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:43.517529 26083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:43.518697 26083 hybrid_clock.cc:648] HybridClock initialized: now 1781888323518678 us; error 39 us; skew 500 ppm
I20260619 16:58:43.520128 26083 webserver.cc:460] Webserver started at http://127.20.231.129:43491/ using document root <none> and password file <none>
I20260619 16:58:43.520430 26083 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:43.520506 26083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:43.520661 26083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:43.521754 26083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-0/data/instance:
uuid: "d7b2bf126a794a7399a295c6ecb9ef24"
format_stamp: "Formatted at 2026-06-19 16:58:43 on dist-test-slave-vnbj"
server_key: "87598c120586311924e375724c589808"
server_key_iv: "c52a89451a8be381c007b79b3d02ea86"
server_key_version: "encryptionkey@0"
I20260619 16:58:43.522135 26083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance:
uuid: "d7b2bf126a794a7399a295c6ecb9ef24"
format_stamp: "Formatted at 2026-06-19 16:58:43 on dist-test-slave-vnbj"
server_key: "87598c120586311924e375724c589808"
server_key_iv: "c52a89451a8be381c007b79b3d02ea86"
server_key_version: "encryptionkey@0"
I20260619 16:58:43.523571 26083 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:58:43.524348 26097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:43.524499 26083 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:58:43.524607 26083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-0/wal
uuid: "d7b2bf126a794a7399a295c6ecb9ef24"
format_stamp: "Formatted at 2026-06-19 16:58:43 on dist-test-slave-vnbj"
server_key: "87598c120586311924e375724c589808"
server_key_iv: "c52a89451a8be381c007b79b3d02ea86"
server_key_version: "encryptionkey@0"
I20260619 16:58:43.524683 26083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:43.528250 26083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:43.528553 26083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:43.528694 26083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:43.528904 26083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:58:43.529209 26083 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:43.529242 26083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:43.529307 26083 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:43.529333 26083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:43.535409 26083 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.129:39445
I20260619 16:58:43.535480 26210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.129:39445 every 8 connection(s)
I20260619 16:58:43.535815 26083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
I20260619 16:58:43.538820 26063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:43.536600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:60175 (local address 127.20.231.190:33769)
0619 16:58:43.536684 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:43.536687 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:43.536936 (+ 249us) server_negotiation.cc:407] Connection header received
0619 16:58:43.537426 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:43.537428 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:43.537433 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:43.537493 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:43.538618 (+ 1125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:43.538619 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:43.538620 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:58:43.538623 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:43.538638 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:58:43.538642 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:43.538688 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:43.538753 (+ 65us) server_negotiation.cc:299] Negotiation successful
0619 16:58:43.538765 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 16:58:43.539084 26213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:43.536567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33769 (local address 127.20.231.129:60175)
0619 16:58:43.536771 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:43.536842 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:43.537299 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:43.537573 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:43.537576 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:43.537585 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:43.538471 (+ 886us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 16:58:43.538471 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:43.538500 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 16:58:43.538528 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:43.538686 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:43.538689 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:43.538691 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 16:58:43.538743 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 16:58:43.538816 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":105,"threads_started":1}
I20260619 16:58:43.539610 26211 heartbeater.cc:347] Connected to a master server at 127.20.231.190:33769
I20260619 16:58:43.539716 26211 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:43.539894 26211 heartbeater.cc:511] Master 127.20.231.190:33769 requested a full tablet report, sending...
I20260619 16:58:43.539956 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 26083
I20260619 16:58:43.540072 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance
I20260619 16:58:43.540261 21406 external_mini_cluster.cc:1428] Setting key ad73a6382fac1b330ec95f586672b222
I20260619 16:58:43.540472 26023 ts_manager.cc:194] Registered new tserver with Master: d7b2bf126a794a7399a295c6ecb9ef24 (127.20.231.129:39445)
I20260619 16:58:43.541357 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.130:0
--local_ip_for_outbound_sockets=127.20.231.130
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:33769
--builtin_ntp_servers=127.20.231.148:40761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 16:58:43.624792 26214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:43.624998 26214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:43.625021 26214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:43.626206 26214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:43.626251 26214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:43.626277 26214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.130
I20260619 16:58:43.627852 26214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:40761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:33769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26214
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.130
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:43.628103 26214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:43.628438 26214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:43.634459 26219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:43.634598 26222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:43.634459 26220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:43.634905 26214 server_base.cc:1034] running on GCE node
I20260619 16:58:43.635159 26214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:43.635447 26214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:43.636571 26214 hybrid_clock.cc:648] HybridClock initialized: now 1781888323636551 us; error 37 us; skew 500 ppm
I20260619 16:58:43.638195 26214 webserver.cc:460] Webserver started at http://127.20.231.130:43913/ using document root <none> and password file <none>
I20260619 16:58:43.638458 26214 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:43.638531 26214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:43.638674 26214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:43.640134 26214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-1/data/instance:
uuid: "128dd14c28cf466a996715606dd3b1f4"
format_stamp: "Formatted at 2026-06-19 16:58:43 on dist-test-slave-vnbj"
server_key: "b7945987f0bd12fd4b47d5a3372cd698"
server_key_iv: "20eb3deba0d537653092e0857c7280c0"
server_key_version: "encryptionkey@0"
I20260619 16:58:43.640578 26214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance:
uuid: "128dd14c28cf466a996715606dd3b1f4"
format_stamp: "Formatted at 2026-06-19 16:58:43 on dist-test-slave-vnbj"
server_key: "b7945987f0bd12fd4b47d5a3372cd698"
server_key_iv: "20eb3deba0d537653092e0857c7280c0"
server_key_version: "encryptionkey@0"
I20260619 16:58:43.642113 26214 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:58:43.642889 26228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:43.643071 26214 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:43.643170 26214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-1/wal
uuid: "128dd14c28cf466a996715606dd3b1f4"
format_stamp: "Formatted at 2026-06-19 16:58:43 on dist-test-slave-vnbj"
server_key: "b7945987f0bd12fd4b47d5a3372cd698"
server_key_iv: "20eb3deba0d537653092e0857c7280c0"
server_key_version: "encryptionkey@0"
I20260619 16:58:43.643263 26214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:43.646633 26214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:43.646956 26214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:43.647104 26214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:43.647320 26214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:58:43.647646 26214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:43.647702 26214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:43.647742 26214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:43.647770 26214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:43.653908 26214 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.130:38149
I20260619 16:58:43.653985 26341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.130:38149 every 8 connection(s)
I20260619 16:58:43.654327 26214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
I20260619 16:58:43.656000 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 26214
I20260619 16:58:43.656095 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance
I20260619 16:58:43.656335 21406 external_mini_cluster.cc:1428] Setting key 9dbe73adda9738d7616dff891d06fcb2
I20260619 16:58:43.657585 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.131:0
--local_ip_for_outbound_sockets=127.20.231.131
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:33769
--builtin_ntp_servers=127.20.231.148:40761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260619 16:58:43.657735 26063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:43.655134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:49963 (local address 127.20.231.190:33769)
0619 16:58:43.655295 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:43.655297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:43.655446 (+ 149us) server_negotiation.cc:407] Connection header received
0619 16:58:43.655905 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:43.655907 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:43.655914 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:43.656008 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:43.657468 (+ 1460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:43.657470 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:43.657471 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:58:43.657476 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:43.657495 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:58:43.657500 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:43.657583 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:43.657635 (+ 52us) server_negotiation.cc:299] Negotiation successful
0619 16:58:43.657651 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260619 16:58:43.657945 26344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:43.655112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33769 (local address 127.20.231.130:49963)
0619 16:58:43.655311 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:43.655351 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:43.655803 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:43.656083 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:43.656088 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:43.656097 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:43.657295 (+ 1198us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 16:58:43.657295 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:43.657326 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 16:58:43.657355 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:43.657579 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:43.657581 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:43.657584 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:58:43.657635 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 16:58:43.657681 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":98,"threads_started":1}
I20260619 16:58:43.658344 26342 heartbeater.cc:347] Connected to a master server at 127.20.231.190:33769
I20260619 16:58:43.658449 26342 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:43.658623 26342 heartbeater.cc:511] Master 127.20.231.190:33769 requested a full tablet report, sending...
I20260619 16:58:43.659192 26022 ts_manager.cc:194] Registered new tserver with Master: 128dd14c28cf466a996715606dd3b1f4 (127.20.231.130:38149)
W20260619 16:58:43.738569 26345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:43.738827 26345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:43.738888 26345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:43.740059 26345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:43.740127 26345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:43.740170 26345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.131
I20260619 16:58:43.741530 26345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:40761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:33769
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.131
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:43.741817 26345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:43.742122 26345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:43.747262 26350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:43.747246 26353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:43.747359 26345 server_base.cc:1034] running on GCE node
W20260619 16:58:43.747273 26351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:43.747642 26345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:43.747895 26345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:43.749056 26345 hybrid_clock.cc:648] HybridClock initialized: now 1781888323749052 us; error 59 us; skew 500 ppm
I20260619 16:58:43.750348 26345 webserver.cc:460] Webserver started at http://127.20.231.131:36991/ using document root <none> and password file <none>
I20260619 16:58:43.750598 26345 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:43.750669 26345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:43.750830 26345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:43.751935 26345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-2/data/instance:
uuid: "557ce1e875784944ada72a00f811d721"
format_stamp: "Formatted at 2026-06-19 16:58:43 on dist-test-slave-vnbj"
server_key: "13f85fa277c24345005b650bb95410e5"
server_key_iv: "40e8210cc91df9ea6f531ed9dca26700"
server_key_version: "encryptionkey@0"
I20260619 16:58:43.752302 26345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance:
uuid: "557ce1e875784944ada72a00f811d721"
format_stamp: "Formatted at 2026-06-19 16:58:43 on dist-test-slave-vnbj"
server_key: "13f85fa277c24345005b650bb95410e5"
server_key_iv: "40e8210cc91df9ea6f531ed9dca26700"
server_key_version: "encryptionkey@0"
I20260619 16:58:43.753659 26345 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:58:43.754407 26359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:43.754573 26345 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:58:43.754678 26345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-2/wal
uuid: "557ce1e875784944ada72a00f811d721"
format_stamp: "Formatted at 2026-06-19 16:58:43 on dist-test-slave-vnbj"
server_key: "13f85fa277c24345005b650bb95410e5"
server_key_iv: "40e8210cc91df9ea6f531ed9dca26700"
server_key_version: "encryptionkey@0"
I20260619 16:58:43.754753 26345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:43.758103 26345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:43.758381 26345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:43.758530 26345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:43.758734 26345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:58:43.759076 26345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:43.759133 26345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:43.759172 26345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:43.759198 26345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:43.765075 26345 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.131:33299
I20260619 16:58:43.765112 26472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.131:33299 every 8 connection(s)
I20260619 16:58:43.765457 26345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
I20260619 16:58:43.768793 26063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:43.766269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:40717 (local address 127.20.231.190:33769)
0619 16:58:43.766381 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:43.766383 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:43.766596 (+ 213us) server_negotiation.cc:407] Connection header received
0619 16:58:43.767177 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:43.767179 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:43.767185 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:43.767232 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:43.768509 (+ 1277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:43.768511 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:43.768512 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:58:43.768517 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:43.768534 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:58:43.768543 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:43.768610 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:43.768706 (+ 96us) server_negotiation.cc:299] Negotiation successful
0619 16:58:43.768722 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 16:58:43.768987 26475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:43.766238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33769 (local address 127.20.231.131:40717)
0619 16:58:43.766472 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:43.766505 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:43.767068 (+ 563us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:43.767300 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:43.767304 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:43.767314 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:43.768349 (+ 1035us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 16:58:43.768349 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:43.768378 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 16:58:43.768407 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:43.768609 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:43.768615 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:43.768618 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:58:43.768717 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 16:58:43.768776 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":91,"threads_started":1}
I20260619 16:58:43.769462 26473 heartbeater.cc:347] Connected to a master server at 127.20.231.190:33769
I20260619 16:58:43.769538 26473 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:43.769716 26473 heartbeater.cc:511] Master 127.20.231.190:33769 requested a full tablet report, sending...
I20260619 16:58:43.770063 26022 ts_manager.cc:194] Registered new tserver with Master: 557ce1e875784944ada72a00f811d721 (127.20.231.131:33299)
I20260619 16:58:43.773052 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 26345
I20260619 16:58:43.773139 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance
I20260619 16:58:43.773319 21406 external_mini_cluster.cc:1428] Setting key 39d275885de8696f2a714f21937e3acf
I20260619 16:58:43.774310 21406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 16:58:43.776769 26063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:43.775756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49242 (local address 127.20.231.190:33769)
0619 16:58:43.775819 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:43.775821 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:43.775965 (+ 144us) server_negotiation.cc:407] Connection header received
0619 16:58:43.776005 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:43.776006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:43.776010 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:43.776063 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:43.776553 (+ 490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:43.776555 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:43.776555 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:58:43.776559 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:43.776571 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:58:43.776575 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:43.776622 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:43.776657 (+ 35us) server_negotiation.cc:299] Negotiation successful
0619 16:58:43.776670 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 16:58:43.778592 26022 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49242:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:58:43.780174 26022 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:58:43.784368 26486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:43.782691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56384 (local address 127.20.231.129:39445)
0619 16:58:43.782912 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:43.782917 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:43.782952 (+ 35us) server_negotiation.cc:407] Connection header received
0619 16:58:43.782979 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:43.782981 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:43.782989 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:43.783307 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:43.783952 (+ 645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:43.783961 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:43.783962 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:58:43.783967 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:43.784003 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:58:43.784041 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:43.784136 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:43.784257 (+ 121us) server_negotiation.cc:299] Negotiation successful
0619 16:58:43.784275 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":89,"threads_started":1}
I20260619 16:58:43.784436 26481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:43.782247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:38149 (local address 127.0.0.1:54998)
0619 16:58:43.782384 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:43.782423 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:43.782476 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:43.783101 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:43.783102 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:43.783107 (+ 5us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:43.783963 (+ 856us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 16:58:43.783964 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:43.783972 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 16:58:43.783984 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:43.784263 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:43.784264 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:43.784265 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 16:58:43.784315 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 16:58:43.784327 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":88,"threads_started":1}
I20260619 16:58:43.784554 26484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:43.782475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54998 (local address 127.20.231.130:38149)
0619 16:58:43.782935 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:43.782938 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:43.782952 (+ 14us) server_negotiation.cc:407] Connection header received
0619 16:58:43.782979 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:43.782981 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:43.782989 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:43.783315 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:43.784104 (+ 789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:43.784110 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:43.784111 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:58:43.784115 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:43.784164 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:58:43.784189 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:43.784415 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:43.784440 (+ 25us) server_negotiation.cc:299] Negotiation successful
0619 16:58:43.784457 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":92,"threads_started":1}
I20260619 16:58:43.784373 26483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:43.782476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:39445 (local address 127.0.0.1:56384)
0619 16:58:43.782591 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:43.782601 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:43.782637 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:43.783102 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:43.783102 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:43.783105 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:43.783783 (+ 678us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 16:58:43.783784 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:43.783804 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 16:58:43.783834 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:43.784141 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:43.784148 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:43.784152 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:58:43.784246 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 16:58:43.784263 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":77,"threads_started":1}
I20260619 16:58:43.784781 26482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:43.782293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:33299 (local address 127.0.0.1:59044)
0619 16:58:43.782470 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:43.782483 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:43.782550 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:43.783049 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:43.783056 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:43.783063 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:43.784099 (+ 1036us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 16:58:43.784100 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:43.784107 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 16:58:43.784121 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:43.784659 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:43.784661 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:43.784661 (+ 0us) client_negotiation.cc:769] Sending connection context
0619 16:58:43.784700 (+ 39us) client_negotiation.cc:240] Negotiation successful
0619 16:58:43.784714 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":101,"threads_started":1}
I20260619 16:58:43.785195 26485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:43.782670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59044 (local address 127.20.231.131:33299)
0619 16:58:43.782829 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:43.782835 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:43.782869 (+ 34us) server_negotiation.cc:407] Connection header received
0619 16:58:43.782899 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:43.782902 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:43.782913 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:43.783998 (+ 1085us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:43.784471 (+ 473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:43.784475 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:43.784475 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:58:43.784479 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:43.784515 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:58:43.784533 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:43.784994 (+ 461us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:43.785068 (+ 74us) server_negotiation.cc:299] Negotiation successful
0619 16:58:43.785090 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":85,"threads_started":1}
I20260619 16:58:43.785270 26276 tablet_service.cc:1467] Processing CreateTablet for tablet 634489b1d99a481e8c6944d6a41ad29c (DEFAULT_TABLE table=test-table [id=7663546f1c6a435d813ebc95a3f7e8af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:43.785418 26145 tablet_service.cc:1467] Processing CreateTablet for tablet 634489b1d99a481e8c6944d6a41ad29c (DEFAULT_TABLE table=test-table [id=7663546f1c6a435d813ebc95a3f7e8af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:43.785687 26145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 634489b1d99a481e8c6944d6a41ad29c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:43.785687 26276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 634489b1d99a481e8c6944d6a41ad29c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:43.785848 26407 tablet_service.cc:1467] Processing CreateTablet for tablet 634489b1d99a481e8c6944d6a41ad29c (DEFAULT_TABLE table=test-table [id=7663546f1c6a435d813ebc95a3f7e8af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:43.786281 26407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 634489b1d99a481e8c6944d6a41ad29c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:43.789330 26487 tablet_bootstrap.cc:492] T 634489b1d99a481e8c6944d6a41ad29c P d7b2bf126a794a7399a295c6ecb9ef24: Bootstrap starting.
I20260619 16:58:43.789659 26488 tablet_bootstrap.cc:492] T 634489b1d99a481e8c6944d6a41ad29c P 128dd14c28cf466a996715606dd3b1f4: Bootstrap starting.
I20260619 16:58:43.789924 26489 tablet_bootstrap.cc:492] T 634489b1d99a481e8c6944d6a41ad29c P 557ce1e875784944ada72a00f811d721: Bootstrap starting.
I20260619 16:58:43.790403 26488 tablet_bootstrap.cc:654] T 634489b1d99a481e8c6944d6a41ad29c P 128dd14c28cf466a996715606dd3b1f4: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:43.790463 26487 tablet_bootstrap.cc:654] T 634489b1d99a481e8c6944d6a41ad29c P d7b2bf126a794a7399a295c6ecb9ef24: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:43.790690 26489 tablet_bootstrap.cc:654] T 634489b1d99a481e8c6944d6a41ad29c P 557ce1e875784944ada72a00f811d721: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:43.790730 26488 log.cc:824] T 634489b1d99a481e8c6944d6a41ad29c P 128dd14c28cf466a996715606dd3b1f4: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:43.790927 26487 log.cc:824] T 634489b1d99a481e8c6944d6a41ad29c P d7b2bf126a794a7399a295c6ecb9ef24: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:43.791066 26489 log.cc:824] T 634489b1d99a481e8c6944d6a41ad29c P 557ce1e875784944ada72a00f811d721: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:43.791667 26488 tablet_bootstrap.cc:492] T 634489b1d99a481e8c6944d6a41ad29c P 128dd14c28cf466a996715606dd3b1f4: No bootstrap required, opened a new log
I20260619 16:58:43.791695 26487 tablet_bootstrap.cc:492] T 634489b1d99a481e8c6944d6a41ad29c P d7b2bf126a794a7399a295c6ecb9ef24: No bootstrap required, opened a new log
I20260619 16:58:43.791756 26488 ts_tablet_manager.cc:1397] T 634489b1d99a481e8c6944d6a41ad29c P 128dd14c28cf466a996715606dd3b1f4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:58:43.791777 26487 ts_tablet_manager.cc:1397] T 634489b1d99a481e8c6944d6a41ad29c P d7b2bf126a794a7399a295c6ecb9ef24: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260619 16:58:43.791795 26489 tablet_bootstrap.cc:492] T 634489b1d99a481e8c6944d6a41ad29c P 557ce1e875784944ada72a00f811d721: No bootstrap required, opened a new log
I20260619 16:58:43.791858 26489 ts_tablet_manager.cc:1397] T 634489b1d99a481e8c6944d6a41ad29c P 557ce1e875784944ada72a00f811d721: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:58:43.793654 26487 raft_consensus.cc:348] T 634489b1d99a481e8c6944d6a41ad29c P d7b2bf126a794a7399a295c6ecb9ef24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "557ce1e875784944ada72a00f811d721" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 33299 } } peers { permanent_uuid: "128dd14c28cf466a996715606dd3b1f4" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 38149 } } peers { permanent_uuid: "d7b2bf126a794a7399a295c6ecb9ef24" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 39445 } }
I20260619 16:58:43.793824 26487 raft_consensus.cc:374] T 634489b1d99a481e8c6944d6a41ad29c P d7b2bf126a794a7399a295c6ecb9ef24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:43.793870 26487 raft_consensus.cc:729] T 634489b1d99a481e8c6944d6a41ad29c P d7b2bf126a794a7399a295c6ecb9ef24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7b2bf126a794a7399a295c6ecb9ef24, State: Initialized, Role: FOLLOWER
I20260619 16:58:43.793984 26487 consensus_queue.cc:260] T 634489b1d99a481e8c6944d6a41ad29c P d7b2bf126a794a7399a295c6ecb9ef24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "557ce1e875784944ada72a00f811d721" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 33299 } } peers { permanent_uuid: "128dd14c28cf466a996715606dd3b1f4" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 38149 } } peers { permanent_uuid: "d7b2bf126a794a7399a295c6ecb9ef24" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 39445 } }
I20260619 16:58:43.794309 26211 heartbeater.cc:503] Master 127.20.231.190:33769 was elected leader, sending a full tablet report...
I20260619 16:58:43.794298 26487 ts_tablet_manager.cc:1428] T 634489b1d99a481e8c6944d6a41ad29c P d7b2bf126a794a7399a295c6ecb9ef24: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260619 16:58:43.794926 26489 raft_consensus.cc:348] T 634489b1d99a481e8c6944d6a41ad29c P 557ce1e875784944ada72a00f811d721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "557ce1e875784944ada72a00f811d721" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 33299 } } peers { permanent_uuid: "128dd14c28cf466a996715606dd3b1f4" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 38149 } } peers { permanent_uuid: "d7b2bf126a794a7399a295c6ecb9ef24" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 39445 } }
I20260619 16:58:43.794919 26488 raft_consensus.cc:348] T 634489b1d99a481e8c6944d6a41ad29c P 128dd14c28cf466a996715606dd3b1f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "557ce1e875784944ada72a00f811d721" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 33299 } } peers { permanent_uuid: "128dd14c28cf466a996715606dd3b1f4" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 38149 } } peers { permanent_uuid: "d7b2bf126a794a7399a295c6ecb9ef24" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 39445 } }
I20260619 16:58:43.795120 26489 raft_consensus.cc:374] T 634489b1d99a481e8c6944d6a41ad29c P 557ce1e875784944ada72a00f811d721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:43.795120 26488 raft_consensus.cc:374] T 634489b1d99a481e8c6944d6a41ad29c P 128dd14c28cf466a996715606dd3b1f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:43.795171 26488 raft_consensus.cc:729] T 634489b1d99a481e8c6944d6a41ad29c P 128dd14c28cf466a996715606dd3b1f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 128dd14c28cf466a996715606dd3b1f4, State: Initialized, Role: FOLLOWER
I20260619 16:58:43.795171 26489 raft_consensus.cc:729] T 634489b1d99a481e8c6944d6a41ad29c P 557ce1e875784944ada72a00f811d721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 557ce1e875784944ada72a00f811d721, State: Initialized, Role: FOLLOWER
I20260619 16:58:43.795300 26488 consensus_queue.cc:260] T 634489b1d99a481e8c6944d6a41ad29c P 128dd14c28cf466a996715606dd3b1f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "557ce1e875784944ada72a00f811d721" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 33299 } } peers { permanent_uuid: "128dd14c28cf466a996715606dd3b1f4" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 38149 } } peers { permanent_uuid: "d7b2bf126a794a7399a295c6ecb9ef24" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 39445 } }
I20260619 16:58:43.795300 26489 consensus_queue.cc:260] T 634489b1d99a481e8c6944d6a41ad29c P 557ce1e875784944ada72a00f811d721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "557ce1e875784944ada72a00f811d721" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 33299 } } peers { permanent_uuid: "128dd14c28cf466a996715606dd3b1f4" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 38149 } } peers { permanent_uuid: "d7b2bf126a794a7399a295c6ecb9ef24" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 39445 } }
I20260619 16:58:43.795599 26489 ts_tablet_manager.cc:1428] T 634489b1d99a481e8c6944d6a41ad29c P 557ce1e875784944ada72a00f811d721: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260619 16:58:43.795756 26473 heartbeater.cc:503] Master 127.20.231.190:33769 was elected leader, sending a full tablet report...
I20260619 16:58:43.795778 26488 ts_tablet_manager.cc:1428] T 634489b1d99a481e8c6944d6a41ad29c P 128dd14c28cf466a996715606dd3b1f4: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260619 16:58:43.796067 26342 heartbeater.cc:503] Master 127.20.231.190:33769 was elected leader, sending a full tablet report...
I20260619 16:58:43.903848 26494 raft_consensus.cc:482] T 634489b1d99a481e8c6944d6a41ad29c P 557ce1e875784944ada72a00f811d721 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:58:43.904007 26494 raft_consensus.cc:504] T 634489b1d99a481e8c6944d6a41ad29c P 557ce1e875784944ada72a00f811d721 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "557ce1e875784944ada72a00f811d721" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 33299 } } peers { permanent_uuid: "128dd14c28cf466a996715606dd3b1f4" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 38149 } } peers { permanent_uuid: "d7b2bf126a794a7399a295c6ecb9ef24" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 39445 } }
I20260619 16:58:43.904397 26494 leader_election.cc:288] T 634489b1d99a481e8c6944d6a41ad29c P 557ce1e875784944ada72a00f811d721 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 128dd14c28cf466a996715606dd3b1f4 (127.20.231.130:38149), d7b2bf126a794a7399a295c6ecb9ef24 (127.20.231.129:39445)
W20260619 16:58:43.904961 26343 tablet.cc:2369] T 634489b1d99a481e8c6944d6a41ad29c P 128dd14c28cf466a996715606dd3b1f4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:58:43.906061 26484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:43.904812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:41115 (local address 127.20.231.130:38149)
0619 16:58:43.904865 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:43.904868 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:43.904880 (+ 12us) server_negotiation.cc:407] Connection header received
0619 16:58:43.904908 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:43.904909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:43.904914 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:43.904973 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:43.905748 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:43.905749 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:43.905750 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:58:43.905755 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:43.905772 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:58:43.905776 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:43.905874 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:43.905970 (+ 96us) server_negotiation.cc:299] Negotiation successful
0619 16:58:43.905987 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 16:58:43.906049 26475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:43.904594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:38149 (local address 127.20.231.131:41115)
0619 16:58:43.904664 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:43.904680 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:43.904740 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:43.904994 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:43.904995 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:43.904997 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:43.905623 (+ 626us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 16:58:43.905624 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:43.905633 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 16:58:43.905649 (+ 16us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:43.905881 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:43.905882 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:43.905883 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 16:58:43.905944 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 16:58:43.905960 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 16:58:43.906140 26486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:43.904851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:46581 (local address 127.20.231.129:39445)
0619 16:58:43.904904 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:43.904906 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:43.904917 (+ 11us) server_negotiation.cc:407] Connection header received
0619 16:58:43.904948 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:43.904949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:43.904955 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:43.905019 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:43.905874 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:43.905876 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:43.905876 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:58:43.905882 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:43.905901 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:58:43.905906 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:43.905970 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:43.906036 (+ 66us) server_negotiation.cc:299] Negotiation successful
0619 16:58:43.906049 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 16:58:43.906172 26496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:43.904695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:39445 (local address 127.20.231.131:46581)
0619 16:58:43.904852 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:43.904862 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:43.904898 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:43.905044 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:43.905045 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:43.905048 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:43.905738 (+ 690us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 16:58:43.905738 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:43.905745 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 16:58:43.905758 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:43.905975 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:43.905976 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:43.905977 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 16:58:43.906028 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 16:58:43.906040 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":100,"threads_started":1}
I20260619 16:58:43.906440 26296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "634489b1d99a481e8c6944d6a41ad29c" candidate_uuid: "557ce1e875784944ada72a00f811d721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "128dd14c28cf466a996715606dd3b1f4" is_pre_election: true
I20260619 16:58:43.906493 26165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "634489b1d99a481e8c6944d6a41ad29c" candidate_uuid: "557ce1e875784944ada72a00f811d721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7b2bf126a794a7399a295c6ecb9ef24" is_pre_election: true
I20260619 16:58:43.906599 26296 raft_consensus.cc:2454] T 634489b1d99a481e8c6944d6a41ad29c P 128dd14c28cf466a996715606dd3b1f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 557ce1e875784944ada72a00f811d721 in term 0.
I20260619 16:58:43.906646 26165 raft_consensus.cc:2454] T 634489b1d99a481e8c6944d6a41ad29c P d7b2bf126a794a7399a295c6ecb9ef24 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 557ce1e875784944ada72a00f811d721 in term 0.
I20260619 16:58:43.906800 26361 leader_election.cc:302] T 634489b1d99a481e8c6944d6a41ad29c P 557ce1e875784944ada72a00f811d721 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 128dd14c28cf466a996715606dd3b1f4, 557ce1e875784944ada72a00f811d721; no voters:
I20260619 16:58:43.906960 26494 raft_consensus.cc:2781] T 634489b1d99a481e8c6944d6a41ad29c P 557ce1e875784944ada72a00f811d721 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:58:43.907013 26494 raft_consensus.cc:482] T 634489b1d99a481e8c6944d6a41ad29c P 557ce1e875784944ada72a00f811d721 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:58:43.907044 26494 raft_consensus.cc:3037] T 634489b1d99a481e8c6944d6a41ad29c P 557ce1e875784944ada72a00f811d721 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:43.907737 26494 raft_consensus.cc:504] T 634489b1d99a481e8c6944d6a41ad29c P 557ce1e875784944ada72a00f811d721 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "557ce1e875784944ada72a00f811d721" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 33299 } } peers { permanent_uuid: "128dd14c28cf466a996715606dd3b1f4" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 38149 } } peers { permanent_uuid: "d7b2bf126a794a7399a295c6ecb9ef24" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 39445 } }
I20260619 16:58:43.907878 26494 leader_election.cc:288] T 634489b1d99a481e8c6944d6a41ad29c P 557ce1e875784944ada72a00f811d721 [CANDIDATE]: Term 1 election: Requested vote from peers 128dd14c28cf466a996715606dd3b1f4 (127.20.231.130:38149), d7b2bf126a794a7399a295c6ecb9ef24 (127.20.231.129:39445)
I20260619 16:58:43.908077 26296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "634489b1d99a481e8c6944d6a41ad29c" candidate_uuid: "557ce1e875784944ada72a00f811d721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "128dd14c28cf466a996715606dd3b1f4"
I20260619 16:58:43.908183 26296 raft_consensus.cc:3037] T 634489b1d99a481e8c6944d6a41ad29c P 128dd14c28cf466a996715606dd3b1f4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:43.909009 26296 raft_consensus.cc:2454] T 634489b1d99a481e8c6944d6a41ad29c P 128dd14c28cf466a996715606dd3b1f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 557ce1e875784944ada72a00f811d721 in term 1.
I20260619 16:58:43.909144 26165 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "634489b1d99a481e8c6944d6a41ad29c" candidate_uuid: "557ce1e875784944ada72a00f811d721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d7b2bf126a794a7399a295c6ecb9ef24"
I20260619 16:58:43.909189 26361 leader_election.cc:302] T 634489b1d99a481e8c6944d6a41ad29c P 557ce1e875784944ada72a00f811d721 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 128dd14c28cf466a996715606dd3b1f4, 557ce1e875784944ada72a00f811d721; no voters:
I20260619 16:58:43.909279 26494 raft_consensus.cc:2781] T 634489b1d99a481e8c6944d6a41ad29c P 557ce1e875784944ada72a00f811d721 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:43.909245 26165 raft_consensus.cc:3037] T 634489b1d99a481e8c6944d6a41ad29c P d7b2bf126a794a7399a295c6ecb9ef24 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:43.909431 26494 raft_consensus.cc:686] T 634489b1d99a481e8c6944d6a41ad29c P 557ce1e875784944ada72a00f811d721 [term 1 LEADER]: Becoming Leader. State: Replica: 557ce1e875784944ada72a00f811d721, State: Running, Role: LEADER
I20260619 16:58:43.909547 26494 consensus_queue.cc:237] T 634489b1d99a481e8c6944d6a41ad29c P 557ce1e875784944ada72a00f811d721 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "557ce1e875784944ada72a00f811d721" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 33299 } } peers { permanent_uuid: "128dd14c28cf466a996715606dd3b1f4" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 38149 } } peers { permanent_uuid: "d7b2bf126a794a7399a295c6ecb9ef24" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 39445 } }
I20260619 16:58:43.910120 26165 raft_consensus.cc:2454] T 634489b1d99a481e8c6944d6a41ad29c P d7b2bf126a794a7399a295c6ecb9ef24 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 557ce1e875784944ada72a00f811d721 in term 1.
I20260619 16:58:43.910728 26022 catalog_manager.cc:5495] T 634489b1d99a481e8c6944d6a41ad29c P 557ce1e875784944ada72a00f811d721 reported cstate change: term changed from 0 to 1, leader changed from <none> to 557ce1e875784944ada72a00f811d721 (127.20.231.131). New cstate: current_term: 1 leader_uuid: "557ce1e875784944ada72a00f811d721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "557ce1e875784944ada72a00f811d721" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 33299 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "128dd14c28cf466a996715606dd3b1f4" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 38149 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d7b2bf126a794a7399a295c6ecb9ef24" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 39445 } health_report { overall_health: UNKNOWN } } }
I20260619 16:58:43.931718 26485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:43.930248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59050 (local address 127.20.231.131:33299)
0619 16:58:43.930315 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:43.930317 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:43.930327 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:58:43.930463 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:43.930465 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:43.930481 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:43.930546 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:43.931296 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:43.931297 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:43.931297 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:58:43.931302 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:43.931321 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:58:43.931325 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:43.931386 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:43.931543 (+ 157us) server_negotiation.cc:299] Negotiation successful
0619 16:58:43.931561 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 16:58:43.933779 26165 raft_consensus.cc:1261] T 634489b1d99a481e8c6944d6a41ad29c P d7b2bf126a794a7399a295c6ecb9ef24 [term 1 FOLLOWER]: Refusing update from remote peer 557ce1e875784944ada72a00f811d721: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:43.933779 26296 raft_consensus.cc:1261] T 634489b1d99a481e8c6944d6a41ad29c P 128dd14c28cf466a996715606dd3b1f4 [term 1 FOLLOWER]: Refusing update from remote peer 557ce1e875784944ada72a00f811d721: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:43.934201 26497 consensus_queue.cc:1035] T 634489b1d99a481e8c6944d6a41ad29c P 557ce1e875784944ada72a00f811d721 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d7b2bf126a794a7399a295c6ecb9ef24" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 39445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:43.934335 26494 consensus_queue.cc:1035] T 634489b1d99a481e8c6944d6a41ad29c P 557ce1e875784944ada72a00f811d721 [LEADER]: Connected to new peer: Peer: permanent_uuid: "128dd14c28cf466a996715606dd3b1f4" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 38149 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:43.936715 26500 mvcc.cc:205] Tried to move back new op lower bound from 7298614574829473792 to 7298614574734258176. Current Snapshot: MvccSnapshot[applied={T|T < 7298614574829473792}]
I20260619 16:58:43.937175 26502 mvcc.cc:205] Tried to move back new op lower bound from 7298614574829473792 to 7298614574734258176. Current Snapshot: MvccSnapshot[applied={T|T < 7298614574829473792}]
I20260619 16:58:43.937345 26501 mvcc.cc:205] Tried to move back new op lower bound from 7298614574829473792 to 7298614574734258176. Current Snapshot: MvccSnapshot[applied={T|T < 7298614574829473792}]
I20260619 16:58:43.938946 26023 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:58:43.939047 26023 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49242:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:58:43.939782 26023 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 04fcc4c6a92f469da5328d3f1449aa7c: Sending DeleteTablet for 3 replicas of tablet 634489b1d99a481e8c6944d6a41ad29c
I20260619 16:58:43.940125 26145 tablet_service.cc:1514] Processing DeleteTablet for tablet 634489b1d99a481e8c6944d6a41ad29c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:43 UTC) from {username='slave'} at 127.0.0.1:56384
I20260619 16:58:43.940135 26407 tablet_service.cc:1514] Processing DeleteTablet for tablet 634489b1d99a481e8c6944d6a41ad29c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:43 UTC) from {username='slave'} at 127.0.0.1:59044
I20260619 16:58:43.940193 26276 tablet_service.cc:1514] Processing DeleteTablet for tablet 634489b1d99a481e8c6944d6a41ad29c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:43 UTC) from {username='slave'} at 127.0.0.1:54998
I20260619 16:58:43.940420 26511 tablet_replica.cc:330] T 634489b1d99a481e8c6944d6a41ad29c P d7b2bf126a794a7399a295c6ecb9ef24: stopping tablet replica
I20260619 16:58:43.940492 26512 tablet_replica.cc:330] T 634489b1d99a481e8c6944d6a41ad29c P 128dd14c28cf466a996715606dd3b1f4: stopping tablet replica
I20260619 16:58:43.940599 26512 raft_consensus.cc:2229] T 634489b1d99a481e8c6944d6a41ad29c P 128dd14c28cf466a996715606dd3b1f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:58:43.940629 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 26083
I20260619 16:58:43.940706 26512 raft_consensus.cc:2258] T 634489b1d99a481e8c6944d6a41ad29c P 128dd14c28cf466a996715606dd3b1f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:43.940615 26511 raft_consensus.cc:2229] T 634489b1d99a481e8c6944d6a41ad29c P d7b2bf126a794a7399a295c6ecb9ef24 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:58:43.940927 26511 raft_consensus.cc:2258] T 634489b1d99a481e8c6944d6a41ad29c P d7b2bf126a794a7399a295c6ecb9ef24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:43.941133 26512 ts_tablet_manager.cc:1905] T 634489b1d99a481e8c6944d6a41ad29c P 128dd14c28cf466a996715606dd3b1f4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:43.942996 26512 ts_tablet_manager.cc:1918] T 634489b1d99a481e8c6944d6a41ad29c P 128dd14c28cf466a996715606dd3b1f4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:58:43.943066 26512 log.cc:1196] T 634489b1d99a481e8c6944d6a41ad29c P 128dd14c28cf466a996715606dd3b1f4: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-1/wal/wals/634489b1d99a481e8c6944d6a41ad29c
I20260619 16:58:43.943348 26512 ts_tablet_manager.cc:1939] T 634489b1d99a481e8c6944d6a41ad29c P 128dd14c28cf466a996715606dd3b1f4: Deleting consensus metadata
I20260619 16:58:43.943892 26008 catalog_manager.cc:4841] TS 128dd14c28cf466a996715606dd3b1f4 (127.20.231.130:38149): tablet 634489b1d99a481e8c6944d6a41ad29c (table test-table [id=7663546f1c6a435d813ebc95a3f7e8af]) successfully deleted
W20260619 16:58:43.948693 26008 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.231.129:39445 (error 108)
I20260619 16:58:43.948925 26510 tablet_replica.cc:330] T 634489b1d99a481e8c6944d6a41ad29c P 557ce1e875784944ada72a00f811d721: stopping tablet replica
I20260619 16:58:43.948973 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 26214
I20260619 16:58:43.949041 26510 raft_consensus.cc:2229] T 634489b1d99a481e8c6944d6a41ad29c P 557ce1e875784944ada72a00f811d721 [term 1 LEADER]: Raft consensus shutting down.
I20260619 16:58:43.951273 26510 raft_consensus.cc:2258] T 634489b1d99a481e8c6944d6a41ad29c P 557ce1e875784944ada72a00f811d721 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:43.951761 26510 ts_tablet_manager.cc:1905] T 634489b1d99a481e8c6944d6a41ad29c P 557ce1e875784944ada72a00f811d721: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:43.953792 26510 ts_tablet_manager.cc:1918] T 634489b1d99a481e8c6944d6a41ad29c P 557ce1e875784944ada72a00f811d721: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:58:43.953866 26510 log.cc:1196] T 634489b1d99a481e8c6944d6a41ad29c P 557ce1e875784944ada72a00f811d721: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781888301345144-21406-0/minicluster-data/ts-2/wal/wals/634489b1d99a481e8c6944d6a41ad29c
I20260619 16:58:43.954146 26510 ts_tablet_manager.cc:1939] T 634489b1d99a481e8c6944d6a41ad29c P 557ce1e875784944ada72a00f811d721: Deleting consensus metadata
I20260619 16:58:43.954162 26482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:43.954021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:39445 (local address 127.0.0.1:56388)
0619 16:58:43.954073 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:43.954125 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.129:39445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260619 16:58:43.954385 26008 catalog_manager.cc:4584] TS d7b2bf126a794a7399a295c6ecb9ef24 (127.20.231.129:39445): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 634489b1d99a481e8c6944d6a41ad29c: Network error: Client connection negotiation failed: client connection to 127.20.231.129:39445: connect: Connection refused (error 111)
I20260619 16:58:43.954658 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 26345
I20260619 16:58:43.954787 26010 catalog_manager.cc:4841] TS 557ce1e875784944ada72a00f811d721 (127.20.231.131:33299): tablet 634489b1d99a481e8c6944d6a41ad29c (table test-table [id=7663546f1c6a435d813ebc95a3f7e8af]) successfully deleted
I20260619 16:58:43.959271 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 25992
2026-06-19T16:58:43Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (719 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260619 16:58:43.978996 21406 mini_oidc.cc:152] Starting JWKS server
I20260619 16:58:43.980037 21406 webserver.cc:460] Webserver started at https://127.0.0.1:43359/ using document root <none> and password file <none>
I20260619 16:58:43.980121 21406 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 16:58:43.980283 21406 webserver.cc:460] Webserver started at http://127.0.0.1:41559/ using document root <none> and password file <none>
2026-06-19T16:58:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:58:43Z Disabled control of system clock
I20260619 16:58:43.988524 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:37023
--webserver_interface=127.20.231.190
--webserver_port=0
--builtin_ntp_servers=127.20.231.148:38487
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:37023
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43359/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888301345144-21406-0/testchainca.cert with env {}
W20260619 16:58:44.071022 26519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:44.071301 26519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:44.071342 26519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:44.072561 26519 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:44.072641 26519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:44.072664 26519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:44.072697 26519 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:44.072724 26519 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 16:58:44.072744 26519 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 16:58:44.072765 26519 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43359/jwks.json
I20260619 16:58:44.074491 26519 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:38487
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:37023
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:37023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43359/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888301345144-21406-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:44.074787 26519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:44.075035 26519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 16:58:44.080194 26519 server_base.cc:1034] running on GCE node
W20260619 16:58:44.080134 26527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:44.080128 26524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:44.080132 26525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:44.080680 26519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:44.081012 26519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:44.082180 26519 hybrid_clock.cc:648] HybridClock initialized: now 1781888324082151 us; error 45 us; skew 500 ppm
I20260619 16:58:44.083596 26519 webserver.cc:460] Webserver started at http://127.20.231.190:41147/ using document root <none> and password file <none>
I20260619 16:58:44.083918 26519 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:44.083971 26519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:44.084144 26519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:44.085291 26519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "8859605a9bed4b24a9db307a12ec0e0f"
format_stamp: "Formatted at 2026-06-19 16:58:44 on dist-test-slave-vnbj"
server_key: "ff89a7daf8c69743f00c3e6dee2d0549"
server_key_iv: "66a6449a23822c1cbfd7f0260f32bda0"
server_key_version: "encryptionkey@0"
I20260619 16:58:44.085668 26519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "8859605a9bed4b24a9db307a12ec0e0f"
format_stamp: "Formatted at 2026-06-19 16:58:44 on dist-test-slave-vnbj"
server_key: "ff89a7daf8c69743f00c3e6dee2d0549"
server_key_iv: "66a6449a23822c1cbfd7f0260f32bda0"
server_key_version: "encryptionkey@0"
I20260619 16:58:44.087206 26519 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:58:44.088083 26533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:44.088290 26519 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:44.088408 26519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "8859605a9bed4b24a9db307a12ec0e0f"
format_stamp: "Formatted at 2026-06-19 16:58:44 on dist-test-slave-vnbj"
server_key: "ff89a7daf8c69743f00c3e6dee2d0549"
server_key_iv: "66a6449a23822c1cbfd7f0260f32bda0"
server_key_version: "encryptionkey@0"
I20260619 16:58:44.088491 26519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:44.106823 26519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:44.107239 26519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:44.107402 26519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:44.111374 26519 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:37023
I20260619 16:58:44.111411 26585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:37023 every 8 connection(s)
I20260619 16:58:44.111768 26519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 16:58:44.112730 26586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:44.113242 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 26519
I20260619 16:58:44.113361 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 16:58:44.113538 21406 external_mini_cluster.cc:1428] Setting key d5a38df0d2ecbd69da261447c4072f63
I20260619 16:58:44.115557 26586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8859605a9bed4b24a9db307a12ec0e0f: Bootstrap starting.
I20260619 16:58:44.116158 26586 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8859605a9bed4b24a9db307a12ec0e0f: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:44.116448 26586 log.cc:824] T 00000000000000000000000000000000 P 8859605a9bed4b24a9db307a12ec0e0f: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:44.117123 26586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8859605a9bed4b24a9db307a12ec0e0f: No bootstrap required, opened a new log
I20260619 16:58:44.118934 26586 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8859605a9bed4b24a9db307a12ec0e0f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8859605a9bed4b24a9db307a12ec0e0f" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37023 } }
I20260619 16:58:44.119133 26586 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8859605a9bed4b24a9db307a12ec0e0f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:44.119192 26586 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8859605a9bed4b24a9db307a12ec0e0f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8859605a9bed4b24a9db307a12ec0e0f, State: Initialized, Role: FOLLOWER
I20260619 16:58:44.119225 26589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:44.114083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41956 (local address 127.20.231.190:37023)
0619 16:58:44.114298 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:44.114306 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:44.114337 (+ 31us) server_negotiation.cc:407] Connection header received
0619 16:58:44.114811 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:44.114819 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:44.114953 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:44.115103 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:44.115340 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:44.116355 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:44.117187 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:44.117612 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:44.117772 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:44.118518 (+ 746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:44.118521 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:44.118522 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:58:44.118709 (+ 187us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:44.118741 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:58:44.118752 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:44.118823 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:44.118935 (+ 112us) server_negotiation.cc:299] Negotiation successful
0619 16:58:44.119013 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":76,"threads_started":1}
I20260619 16:58:44.119359 26586 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8859605a9bed4b24a9db307a12ec0e0f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8859605a9bed4b24a9db307a12ec0e0f" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37023 } }
I20260619 16:58:44.119448 26586 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8859605a9bed4b24a9db307a12ec0e0f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:58:44.119473 26586 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8859605a9bed4b24a9db307a12ec0e0f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:58:44.119549 26586 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8859605a9bed4b24a9db307a12ec0e0f [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:44.120111 21406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 16:58:44.120209 26586 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8859605a9bed4b24a9db307a12ec0e0f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8859605a9bed4b24a9db307a12ec0e0f" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37023 } }
I20260619 16:58:44.120317 26586 leader_election.cc:302] T 00000000000000000000000000000000 P 8859605a9bed4b24a9db307a12ec0e0f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8859605a9bed4b24a9db307a12ec0e0f; no voters:
I20260619 16:58:44.120466 26586 leader_election.cc:288] T 00000000000000000000000000000000 P 8859605a9bed4b24a9db307a12ec0e0f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:58:44.120527 26591 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8859605a9bed4b24a9db307a12ec0e0f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:44.120687 26591 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8859605a9bed4b24a9db307a12ec0e0f [term 1 LEADER]: Becoming Leader. State: Replica: 8859605a9bed4b24a9db307a12ec0e0f, State: Running, Role: LEADER
I20260619 16:58:44.120728 26586 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8859605a9bed4b24a9db307a12ec0e0f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:44.120769 26591 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8859605a9bed4b24a9db307a12ec0e0f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8859605a9bed4b24a9db307a12ec0e0f" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37023 } }
I20260619 16:58:44.121076 26591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8859605a9bed4b24a9db307a12ec0e0f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8859605a9bed4b24a9db307a12ec0e0f. Latest consensus state: current_term: 1 leader_uuid: "8859605a9bed4b24a9db307a12ec0e0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8859605a9bed4b24a9db307a12ec0e0f" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37023 } } }
I20260619 16:58:44.121140 26591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8859605a9bed4b24a9db307a12ec0e0f [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:44.121440 26592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8859605a9bed4b24a9db307a12ec0e0f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8859605a9bed4b24a9db307a12ec0e0f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8859605a9bed4b24a9db307a12ec0e0f" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37023 } } }
I20260619 16:58:44.121574 26592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8859605a9bed4b24a9db307a12ec0e0f [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:44.121727 26596 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:58:44.122313 26596 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:58:44.123925 26596 catalog_manager.cc:1347] Generated new cluster ID: a8a527c6d5e743a8b6e5f1b371e59ae3
I20260619 16:58:44.123977 26596 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:58:44.138850 26596 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:58:44.139454 26596 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:58:44.145350 26596 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8859605a9bed4b24a9db307a12ec0e0f: Generated new TSK 0
I20260619 16:58:44.145553 26596 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 16:58:44.172683 26616 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260619 16:58:44.172803 26616 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260619 16:58:44.173287 26615 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:44.157059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:37023 (local address 127.0.0.1:41970)
0619 16:58:44.157291 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:44.157301 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:44.157359 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:44.157539 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:44.157540 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:44.157545 (+ 5us) client_negotiation.cc:189] Negotiated authn=JWT
0619 16:58:44.157600 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:44.157601 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:44.158610 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:44.158611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:44.159379 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:44.159382 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:44.159909 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:44.159910 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:44.160006 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:44.160034 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 16:58:44.172951 (+ 12917us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43359/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 16:58:44.173040 (+ 89us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.231.190:37023: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43359/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":131,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.20.231.190:37023: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43359/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260619 16:58:44.173807 26589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:44.157141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41970 (local address 127.20.231.190:37023)
0619 16:58:44.157209 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:44.157211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:44.157360 (+ 149us) server_negotiation.cc:407] Connection header received
0619 16:58:44.157434 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:44.157435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:44.157448 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:44.157526 (+ 78us) server_negotiation.cc:226] Negotiated authn=JWT
0619 16:58:44.157699 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:44.158511 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:44.159446 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:44.159796 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:44.159868 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:44.160185 (+ 317us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 16:58:44.172791 (+ 12606us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43359/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 16:58:44.173683 (+ 892us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41970: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43359/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 16:58:44.173838 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 26519
W20260619 16:58:44.173878 26589 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41970: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43359/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
2026-06-19T16:58:44Z chronyd exiting
I20260619 16:58:44.209764 21406 test_util.cc:175] -----------------------------------------------
I20260619 16:58:44.209911 21406 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781888301345144-21406-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (232 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260619 16:58:44.210716 21406 mini_oidc.cc:152] Starting JWKS server
I20260619 16:58:44.211476 21406 webserver.cc:460] Webserver started at https://127.0.0.1:41223/ using document root <none> and password file <none>
I20260619 16:58:44.211544 21406 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 16:58:44.211706 21406 webserver.cc:460] Webserver started at http://127.0.0.1:44957/ using document root <none> and password file <none>
2026-06-19T16:58:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:58:44Z Disabled control of system clock
I20260619 16:58:44.219515 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:44983
--webserver_interface=127.20.231.190
--webserver_port=0
--builtin_ntp_servers=127.20.231.148:36387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:44983
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41223/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888301345144-21406-0/testchainca.cert with env {}
W20260619 16:58:44.300261 26622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:44.300495 26622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:44.300519 26622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:44.301685 26622 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:44.301730 26622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:44.301743 26622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:44.301756 26622 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:44.301769 26622 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 16:58:44.301787 26622 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 16:58:44.301800 26622 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41223/jwks.json
I20260619 16:58:44.303388 26622 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:36387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:44983
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:44983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41223/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888301345144-21406-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:44.303610 26622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:44.303776 26622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:44.308822 26628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:44.308889 26630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:44.308830 26627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:44.309144 26622 server_base.cc:1034] running on GCE node
I20260619 16:58:44.309521 26622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:44.309789 26622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:44.310916 26622 hybrid_clock.cc:648] HybridClock initialized: now 1781888324310878 us; error 58 us; skew 500 ppm
I20260619 16:58:44.312208 26622 webserver.cc:460] Webserver started at http://127.20.231.190:34683/ using document root <none> and password file <none>
I20260619 16:58:44.312474 26622 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:44.312520 26622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:44.312613 26622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:44.313701 26622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "ddcc0e3ca7804427872f8eab4a2a98b0"
format_stamp: "Formatted at 2026-06-19 16:58:44 on dist-test-slave-vnbj"
server_key: "e6f2f2713b2e541fd4fde004ad7a2d12"
server_key_iv: "78a458978da9ea1c099f6d34064208a8"
server_key_version: "encryptionkey@0"
I20260619 16:58:44.314019 26622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "ddcc0e3ca7804427872f8eab4a2a98b0"
format_stamp: "Formatted at 2026-06-19 16:58:44 on dist-test-slave-vnbj"
server_key: "e6f2f2713b2e541fd4fde004ad7a2d12"
server_key_iv: "78a458978da9ea1c099f6d34064208a8"
server_key_version: "encryptionkey@0"
I20260619 16:58:44.315367 26622 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:58:44.316118 26636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:44.316327 26622 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:58:44.316412 26622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "ddcc0e3ca7804427872f8eab4a2a98b0"
format_stamp: "Formatted at 2026-06-19 16:58:44 on dist-test-slave-vnbj"
server_key: "e6f2f2713b2e541fd4fde004ad7a2d12"
server_key_iv: "78a458978da9ea1c099f6d34064208a8"
server_key_version: "encryptionkey@0"
I20260619 16:58:44.316468 26622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:44.343336 26622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:44.343691 26622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:44.343816 26622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:44.348143 26622 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:44983
I20260619 16:58:44.348178 26688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:44983 every 8 connection(s)
I20260619 16:58:44.348572 26622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 16:58:44.349316 26689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:44.352147 26689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddcc0e3ca7804427872f8eab4a2a98b0: Bootstrap starting.
I20260619 16:58:44.352731 26689 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ddcc0e3ca7804427872f8eab4a2a98b0: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:44.353026 26689 log.cc:824] T 00000000000000000000000000000000 P ddcc0e3ca7804427872f8eab4a2a98b0: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:44.353757 26689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddcc0e3ca7804427872f8eab4a2a98b0: No bootstrap required, opened a new log
I20260619 16:58:44.354259 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 26622
I20260619 16:58:44.354351 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 16:58:44.354573 21406 external_mini_cluster.cc:1428] Setting key ccd8d85b11047e35fed7ca2e87500738
I20260619 16:58:44.356065 26689 raft_consensus.cc:348] T 00000000000000000000000000000000 P ddcc0e3ca7804427872f8eab4a2a98b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddcc0e3ca7804427872f8eab4a2a98b0" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 44983 } }
I20260619 16:58:44.356246 26689 raft_consensus.cc:374] T 00000000000000000000000000000000 P ddcc0e3ca7804427872f8eab4a2a98b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:44.356293 26689 raft_consensus.cc:729] T 00000000000000000000000000000000 P ddcc0e3ca7804427872f8eab4a2a98b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddcc0e3ca7804427872f8eab4a2a98b0, State: Initialized, Role: FOLLOWER
I20260619 16:58:44.356448 26689 consensus_queue.cc:260] T 00000000000000000000000000000000 P ddcc0e3ca7804427872f8eab4a2a98b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddcc0e3ca7804427872f8eab4a2a98b0" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 44983 } }
I20260619 16:58:44.356559 26689 raft_consensus.cc:388] T 00000000000000000000000000000000 P ddcc0e3ca7804427872f8eab4a2a98b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:58:44.356592 26689 raft_consensus.cc:482] T 00000000000000000000000000000000 P ddcc0e3ca7804427872f8eab4a2a98b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:58:44.356632 26689 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ddcc0e3ca7804427872f8eab4a2a98b0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:44.357446 26689 raft_consensus.cc:504] T 00000000000000000000000000000000 P ddcc0e3ca7804427872f8eab4a2a98b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddcc0e3ca7804427872f8eab4a2a98b0" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 44983 } }
I20260619 16:58:44.357578 26689 leader_election.cc:302] T 00000000000000000000000000000000 P ddcc0e3ca7804427872f8eab4a2a98b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ddcc0e3ca7804427872f8eab4a2a98b0; no voters:
I20260619 16:58:44.357729 26689 leader_election.cc:288] T 00000000000000000000000000000000 P ddcc0e3ca7804427872f8eab4a2a98b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:58:44.357851 26694 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ddcc0e3ca7804427872f8eab4a2a98b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:44.357956 26689 sys_catalog.cc:564] T 00000000000000000000000000000000 P ddcc0e3ca7804427872f8eab4a2a98b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:44.358035 26694 raft_consensus.cc:686] T 00000000000000000000000000000000 P ddcc0e3ca7804427872f8eab4a2a98b0 [term 1 LEADER]: Becoming Leader. State: Replica: ddcc0e3ca7804427872f8eab4a2a98b0, State: Running, Role: LEADER
I20260619 16:58:44.358141 26694 consensus_queue.cc:237] T 00000000000000000000000000000000 P ddcc0e3ca7804427872f8eab4a2a98b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddcc0e3ca7804427872f8eab4a2a98b0" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 44983 } }
I20260619 16:58:44.358527 26695 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddcc0e3ca7804427872f8eab4a2a98b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ddcc0e3ca7804427872f8eab4a2a98b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddcc0e3ca7804427872f8eab4a2a98b0" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 44983 } } }
I20260619 16:58:44.358606 26695 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddcc0e3ca7804427872f8eab4a2a98b0 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:44.358767 26696 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddcc0e3ca7804427872f8eab4a2a98b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ddcc0e3ca7804427872f8eab4a2a98b0. Latest consensus state: current_term: 1 leader_uuid: "ddcc0e3ca7804427872f8eab4a2a98b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddcc0e3ca7804427872f8eab4a2a98b0" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 44983 } } }
I20260619 16:58:44.358829 26696 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddcc0e3ca7804427872f8eab4a2a98b0 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:44.359120 26703 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:58:44.359722 26703 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:58:44.361291 26703 catalog_manager.cc:1347] Generated new cluster ID: 0c5dcd28160b42a8b59ec54b2f5592f7
I20260619 16:58:44.361382 26703 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:58:44.361797 26693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:44.355213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48436 (local address 127.20.231.190:44983)
0619 16:58:44.355444 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:44.355454 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:44.355490 (+ 36us) server_negotiation.cc:407] Connection header received
0619 16:58:44.356016 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:44.356025 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:44.356117 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:44.356278 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:44.356565 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:44.358428 (+ 1863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:44.359631 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:44.360219 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:44.360357 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:44.361109 (+ 752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:44.361113 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:44.361113 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:58:44.361321 (+ 208us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:44.361362 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:58:44.361376 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:44.361462 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:44.361540 (+ 78us) server_negotiation.cc:299] Negotiation successful
0619 16:58:44.361622 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":88,"threads_started":1}
I20260619 16:58:44.362643 21406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 16:58:44.371402 26703 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:58:44.372017 26703 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:58:44.375257 26703 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ddcc0e3ca7804427872f8eab4a2a98b0: Generated new TSK 0
I20260619 16:58:44.375461 26703 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 16:58:44.407804 26719 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260619 16:58:44.407898 26719 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 16:58:44.408214 26693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:44.385900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48442 (local address 127.20.231.190:44983)
0619 16:58:44.385948 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:44.385951 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:44.386034 (+ 83us) server_negotiation.cc:407] Connection header received
0619 16:58:44.386103 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:44.386104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:44.386120 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:44.386183 (+ 63us) server_negotiation.cc:226] Negotiated authn=JWT
0619 16:58:44.386321 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:44.386840 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:44.388058 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:44.388248 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:44.388306 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:44.388490 (+ 184us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 16:58:44.407999 (+ 19509us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41223/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 16:58:44.408158 (+ 159us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48442: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41223/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":25}
W20260619 16:58:44.408285 26693 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48442: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41223/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260619 16:58:44.408365 26718 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:44.385800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:44983 (local address 127.0.0.1:48442)
0619 16:58:44.385956 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:44.385968 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:44.386034 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:44.386194 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:44.386194 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:44.386196 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 16:58:44.386255 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:44.386257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:44.386996 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:44.386998 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:44.387965 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:44.387969 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:44.388327 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:44.388328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:44.388391 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:44.388397 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 16:58:44.408147 (+ 19750us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41223/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 16:58:44.408270 (+ 123us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.231.190:44983: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41223/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":102,"threads_started":1}
I20260619 16:58:44.408788 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 26622
2026-06-19T16:58:44Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (235 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260619 16:58:44.446591 21406 mini_oidc.cc:152] Starting JWKS server
I20260619 16:58:44.447366 21406 webserver.cc:460] Webserver started at https://127.0.0.1:36541/ using document root <none> and password file <none>
I20260619 16:58:44.447414 21406 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 16:58:44.447572 21406 webserver.cc:460] Webserver started at http://127.0.0.1:40331/ using document root <none> and password file <none>
2026-06-19T16:58:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:58:44Z Disabled control of system clock
I20260619 16:58:44.455900 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:42045
--webserver_interface=127.20.231.190
--webserver_port=0
--builtin_ntp_servers=127.20.231.148:38493
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:42045
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36541/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 16:58:44.538962 26725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:44.539222 26725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:44.539260 26725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:44.540520 26725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:44.540587 26725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:44.540616 26725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:44.540650 26725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:44.540674 26725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 16:58:44.540705 26725 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 16:58:44.540731 26725 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36541/jwks.json
I20260619 16:58:44.542132 26725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:38493
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:42045
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:42045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36541/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:44.542423 26725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:44.542737 26725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:44.547704 26731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:44.548094 26730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:44.548169 26733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:44.548308 26725 server_base.cc:1034] running on GCE node
I20260619 16:58:44.548563 26725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:44.548903 26725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:44.550133 26725 hybrid_clock.cc:648] HybridClock initialized: now 1781888324550065 us; error 55 us; skew 500 ppm
I20260619 16:58:44.551600 26725 webserver.cc:460] Webserver started at http://127.20.231.190:41195/ using document root <none> and password file <none>
I20260619 16:58:44.551915 26725 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:44.551993 26725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:44.552146 26725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:44.553339 26725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "dd362c1d8f5543e0808caa18f094a3b9"
format_stamp: "Formatted at 2026-06-19 16:58:44 on dist-test-slave-vnbj"
server_key: "da537ccea342b98ff1d0b7f97ce61770"
server_key_iv: "53c2d77b16255b9732aef33bbf9c647b"
server_key_version: "encryptionkey@0"
I20260619 16:58:44.553705 26725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "dd362c1d8f5543e0808caa18f094a3b9"
format_stamp: "Formatted at 2026-06-19 16:58:44 on dist-test-slave-vnbj"
server_key: "da537ccea342b98ff1d0b7f97ce61770"
server_key_iv: "53c2d77b16255b9732aef33bbf9c647b"
server_key_version: "encryptionkey@0"
I20260619 16:58:44.555151 26725 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:58:44.555984 26739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:44.556169 26725 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:44.556277 26725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "dd362c1d8f5543e0808caa18f094a3b9"
format_stamp: "Formatted at 2026-06-19 16:58:44 on dist-test-slave-vnbj"
server_key: "da537ccea342b98ff1d0b7f97ce61770"
server_key_iv: "53c2d77b16255b9732aef33bbf9c647b"
server_key_version: "encryptionkey@0"
I20260619 16:58:44.556354 26725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:44.567847 26725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:44.568173 26725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:44.568329 26725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:44.572790 26725 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:42045
I20260619 16:58:44.572831 26791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:42045 every 8 connection(s)
I20260619 16:58:44.573206 26725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 16:58:44.574218 26792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:44.576622 26792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd362c1d8f5543e0808caa18f094a3b9: Bootstrap starting.
I20260619 16:58:44.577193 26792 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd362c1d8f5543e0808caa18f094a3b9: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:44.577425 26792 log.cc:824] T 00000000000000000000000000000000 P dd362c1d8f5543e0808caa18f094a3b9: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:44.578104 26792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd362c1d8f5543e0808caa18f094a3b9: No bootstrap required, opened a new log
I20260619 16:58:44.579715 26792 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd362c1d8f5543e0808caa18f094a3b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd362c1d8f5543e0808caa18f094a3b9" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 42045 } }
I20260619 16:58:44.579871 26792 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd362c1d8f5543e0808caa18f094a3b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:44.579895 26792 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd362c1d8f5543e0808caa18f094a3b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd362c1d8f5543e0808caa18f094a3b9, State: Initialized, Role: FOLLOWER
I20260619 16:58:44.580003 26792 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd362c1d8f5543e0808caa18f094a3b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd362c1d8f5543e0808caa18f094a3b9" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 42045 } }
I20260619 16:58:44.580086 26792 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd362c1d8f5543e0808caa18f094a3b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:58:44.580118 26792 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd362c1d8f5543e0808caa18f094a3b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:58:44.580183 26792 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd362c1d8f5543e0808caa18f094a3b9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:44.580672 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 26725
I20260619 16:58:44.580751 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 16:58:44.580789 26792 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd362c1d8f5543e0808caa18f094a3b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd362c1d8f5543e0808caa18f094a3b9" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 42045 } }
I20260619 16:58:44.580914 26792 leader_election.cc:302] T 00000000000000000000000000000000 P dd362c1d8f5543e0808caa18f094a3b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd362c1d8f5543e0808caa18f094a3b9; no voters:
I20260619 16:58:44.580950 21406 external_mini_cluster.cc:1428] Setting key f07956e4896893a5dbfa9dd356cc3d5a
I20260619 16:58:44.581106 26792 leader_election.cc:288] T 00000000000000000000000000000000 P dd362c1d8f5543e0808caa18f094a3b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:58:44.581131 26795 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd362c1d8f5543e0808caa18f094a3b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:44.581252 26795 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd362c1d8f5543e0808caa18f094a3b9 [term 1 LEADER]: Becoming Leader. State: Replica: dd362c1d8f5543e0808caa18f094a3b9, State: Running, Role: LEADER
I20260619 16:58:44.581350 26795 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd362c1d8f5543e0808caa18f094a3b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd362c1d8f5543e0808caa18f094a3b9" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 42045 } }
I20260619 16:58:44.581384 26792 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd362c1d8f5543e0808caa18f094a3b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:44.581720 26796 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd362c1d8f5543e0808caa18f094a3b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd362c1d8f5543e0808caa18f094a3b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd362c1d8f5543e0808caa18f094a3b9" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 42045 } } }
I20260619 16:58:44.582070 26796 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd362c1d8f5543e0808caa18f094a3b9 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:44.581987 26797 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd362c1d8f5543e0808caa18f094a3b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd362c1d8f5543e0808caa18f094a3b9. Latest consensus state: current_term: 1 leader_uuid: "dd362c1d8f5543e0808caa18f094a3b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd362c1d8f5543e0808caa18f094a3b9" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 42045 } } }
I20260619 16:58:44.582336 26797 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd362c1d8f5543e0808caa18f094a3b9 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:44.582425 26806 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:58:44.582993 26806 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:58:44.584610 26806 catalog_manager.cc:1347] Generated new cluster ID: e72414b0860f42f09e1a4709ad4d9182
I20260619 16:58:44.584676 26806 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:58:44.587553 26800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:44.581640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55268 (local address 127.20.231.190:42045)
0619 16:58:44.582504 (+ 864us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:44.582515 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:44.582549 (+ 34us) server_negotiation.cc:407] Connection header received
0619 16:58:44.582677 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:44.582686 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:44.582759 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:44.582970 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:44.583205 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:44.584614 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:44.585487 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:44.585921 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:44.586062 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:44.586799 (+ 737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:44.586803 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:44.586804 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:58:44.587052 (+ 248us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:44.587085 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:58:44.587097 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:44.587174 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:44.587252 (+ 78us) server_negotiation.cc:299] Negotiation successful
0619 16:58:44.587337 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":781,"thread_start_us":90,"threads_started":1}
I20260619 16:58:44.588482 21406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 16:58:44.594240 26806 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:58:44.594758 26806 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:58:44.600456 26806 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd362c1d8f5543e0808caa18f094a3b9: Generated new TSK 0
I20260619 16:58:44.600664 26806 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:58:44.615435 26800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:44.611612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55270 (local address 127.20.231.190:42045)
0619 16:58:44.611679 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:44.611681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:44.611822 (+ 141us) server_negotiation.cc:407] Connection header received
0619 16:58:44.611899 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:44.611900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:44.611910 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:44.611971 (+ 61us) server_negotiation.cc:226] Negotiated authn=JWT
0619 16:58:44.612108 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:44.612615 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:44.613756 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:44.613954 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:44.614007 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:44.614199 (+ 192us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 16:58:44.615194 (+ 995us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36541/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 16:58:44.615359 (+ 165us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55270: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36541/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":39}
W20260619 16:58:44.615499 26800 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55270: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36541/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260619 16:58:44.615510 26821 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:44.611544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:42045 (local address 127.0.0.1:55270)
0619 16:58:44.611740 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:44.611753 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:44.611825 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:44.611985 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:44.611986 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:44.611988 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 16:58:44.612047 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:44.612049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:44.612699 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:44.612699 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:44.613668 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:44.613671 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:44.614030 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:44.614031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:44.614092 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:44.614106 (+ 14us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 16:58:44.615299 (+ 1193us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36541/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 16:58:44.615411 (+ 112us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.231.190:42045: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36541/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":125,"threads_started":1}
I20260619 16:58:44.616055 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 26725
E20260619 16:58:44.626343 26822 webserver.cc:533] Webserver: sslize failed:
E20260619 16:58:44.626454 26822 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-06-19T16:58:44Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (206 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-06-19T16:58:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:58:44Z Disabled control of system clock
I20260619 16:58:44.661377 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:41493
--webserver_interface=127.20.231.190
--webserver_port=0
--builtin_ntp_servers=127.20.231.148:45805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:41493
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888301345144-21406-0/testchainca.cert with env {}
W20260619 16:58:44.742120 26826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:44.742340 26826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:44.742364 26826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:44.743518 26826 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:44.743556 26826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:44.743569 26826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:44.743582 26826 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:44.743594 26826 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 16:58:44.743606 26826 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 16:58:44.743618 26826 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260619 16:58:44.745045 26826 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:45805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:41493
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:41493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888301345144-21406-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26826
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:44.745317 26826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:44.745493 26826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:44.750808 26831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:44.750938 26834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:44.750909 26832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:44.751065 26826 server_base.cc:1034] running on GCE node
I20260619 16:58:44.751327 26826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:44.751636 26826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:44.752779 26826 hybrid_clock.cc:648] HybridClock initialized: now 1781888324752755 us; error 42 us; skew 500 ppm
I20260619 16:58:44.753979 26826 webserver.cc:460] Webserver started at http://127.20.231.190:41341/ using document root <none> and password file <none>
I20260619 16:58:44.754237 26826 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:44.754302 26826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:44.754395 26826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:44.755512 26826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "632bafceed0245c3850d6e63a9562b21"
format_stamp: "Formatted at 2026-06-19 16:58:44 on dist-test-slave-vnbj"
server_key: "66e87a8ded0080b7435b26b288ecbeda"
server_key_iv: "35d510d0d2e91540f6ebdf89df1daf2f"
server_key_version: "encryptionkey@0"
I20260619 16:58:44.755821 26826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "632bafceed0245c3850d6e63a9562b21"
format_stamp: "Formatted at 2026-06-19 16:58:44 on dist-test-slave-vnbj"
server_key: "66e87a8ded0080b7435b26b288ecbeda"
server_key_iv: "35d510d0d2e91540f6ebdf89df1daf2f"
server_key_version: "encryptionkey@0"
I20260619 16:58:44.757040 26826 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:58:44.757763 26840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:44.757949 26826 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:58:44.758015 26826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "632bafceed0245c3850d6e63a9562b21"
format_stamp: "Formatted at 2026-06-19 16:58:44 on dist-test-slave-vnbj"
server_key: "66e87a8ded0080b7435b26b288ecbeda"
server_key_iv: "35d510d0d2e91540f6ebdf89df1daf2f"
server_key_version: "encryptionkey@0"
I20260619 16:58:44.758108 26826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:44.769517 26826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:44.769838 26826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:44.769977 26826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:44.774338 26826 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:41493
I20260619 16:58:44.774379 26892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:41493 every 8 connection(s)
I20260619 16:58:44.774725 26826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 16:58:44.775704 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 26826
I20260619 16:58:44.775755 26893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:44.775825 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 16:58:44.776010 21406 external_mini_cluster.cc:1428] Setting key 4cc250a7c72aaa9d69710c98a2c694f0
I20260619 16:58:44.779312 26893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 632bafceed0245c3850d6e63a9562b21: Bootstrap starting.
I20260619 16:58:44.780162 26893 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 632bafceed0245c3850d6e63a9562b21: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:44.780483 26893 log.cc:824] T 00000000000000000000000000000000 P 632bafceed0245c3850d6e63a9562b21: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:44.781208 26893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 632bafceed0245c3850d6e63a9562b21: No bootstrap required, opened a new log
I20260619 16:58:44.782392 26896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:44.776529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46302 (local address 127.20.231.190:41493)
0619 16:58:44.776825 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:44.776836 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:44.776871 (+ 35us) server_negotiation.cc:407] Connection header received
0619 16:58:44.777392 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:44.777402 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:44.777496 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:44.777662 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:44.777818 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:44.779228 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:44.780370 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:44.780885 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:44.781031 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:44.781775 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:44.781780 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:44.781780 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:58:44.781950 (+ 170us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:44.781981 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:58:44.781992 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:44.782062 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:44.782138 (+ 76us) server_negotiation.cc:299] Negotiation successful
0619 16:58:44.782220 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":104,"threads_started":1}
I20260619 16:58:44.783304 21406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 16:58:44.783325 26893 raft_consensus.cc:348] T 00000000000000000000000000000000 P 632bafceed0245c3850d6e63a9562b21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "632bafceed0245c3850d6e63a9562b21" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41493 } }
I20260619 16:58:44.783486 26893 raft_consensus.cc:374] T 00000000000000000000000000000000 P 632bafceed0245c3850d6e63a9562b21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:44.783522 26893 raft_consensus.cc:729] T 00000000000000000000000000000000 P 632bafceed0245c3850d6e63a9562b21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632bafceed0245c3850d6e63a9562b21, State: Initialized, Role: FOLLOWER
I20260619 16:58:44.783634 26893 consensus_queue.cc:260] T 00000000000000000000000000000000 P 632bafceed0245c3850d6e63a9562b21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "632bafceed0245c3850d6e63a9562b21" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41493 } }
I20260619 16:58:44.783701 26893 raft_consensus.cc:388] T 00000000000000000000000000000000 P 632bafceed0245c3850d6e63a9562b21 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:58:44.783728 26893 raft_consensus.cc:482] T 00000000000000000000000000000000 P 632bafceed0245c3850d6e63a9562b21 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:58:44.783782 26893 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 632bafceed0245c3850d6e63a9562b21 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:44.784403 26893 raft_consensus.cc:504] T 00000000000000000000000000000000 P 632bafceed0245c3850d6e63a9562b21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "632bafceed0245c3850d6e63a9562b21" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41493 } }
I20260619 16:58:44.784489 26893 leader_election.cc:302] T 00000000000000000000000000000000 P 632bafceed0245c3850d6e63a9562b21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 632bafceed0245c3850d6e63a9562b21; no voters:
I20260619 16:58:44.784611 26893 leader_election.cc:288] T 00000000000000000000000000000000 P 632bafceed0245c3850d6e63a9562b21 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:58:44.784682 26898 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 632bafceed0245c3850d6e63a9562b21 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:44.784830 26898 raft_consensus.cc:686] T 00000000000000000000000000000000 P 632bafceed0245c3850d6e63a9562b21 [term 1 LEADER]: Becoming Leader. State: Replica: 632bafceed0245c3850d6e63a9562b21, State: Running, Role: LEADER
I20260619 16:58:44.784881 26893 sys_catalog.cc:564] T 00000000000000000000000000000000 P 632bafceed0245c3850d6e63a9562b21 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:44.784912 26898 consensus_queue.cc:237] T 00000000000000000000000000000000 P 632bafceed0245c3850d6e63a9562b21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "632bafceed0245c3850d6e63a9562b21" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41493 } }
I20260619 16:58:44.785323 26899 sys_catalog.cc:455] T 00000000000000000000000000000000 P 632bafceed0245c3850d6e63a9562b21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "632bafceed0245c3850d6e63a9562b21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "632bafceed0245c3850d6e63a9562b21" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41493 } } }
I20260619 16:58:44.785323 26900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 632bafceed0245c3850d6e63a9562b21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 632bafceed0245c3850d6e63a9562b21. Latest consensus state: current_term: 1 leader_uuid: "632bafceed0245c3850d6e63a9562b21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "632bafceed0245c3850d6e63a9562b21" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41493 } } }
I20260619 16:58:44.785497 26900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 632bafceed0245c3850d6e63a9562b21 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:44.785486 26899 sys_catalog.cc:458] T 00000000000000000000000000000000 P 632bafceed0245c3850d6e63a9562b21 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:44.785869 26907 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:58:44.786350 26907 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:58:44.787972 26907 catalog_manager.cc:1347] Generated new cluster ID: 384c6e891e264f84abc0281ccd90fd0b
I20260619 16:58:44.788029 26907 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:58:44.798339 26907 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:58:44.798990 26907 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:58:44.803896 26907 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 632bafceed0245c3850d6e63a9562b21: Generated new TSK 0
I20260619 16:58:44.804101 26907 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:58:45.258985 26896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:44.810112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46316 (local address 127.20.231.190:41493)
0619 16:58:44.810186 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:44.810188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:44.810317 (+ 129us) server_negotiation.cc:407] Connection header received
0619 16:58:44.810359 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:44.810360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:44.810374 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:44.810459 (+ 85us) server_negotiation.cc:226] Negotiated authn=JWT
0619 16:58:44.810560 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:44.811417 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:44.812218 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:44.812532 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:44.812605 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:44.812757 (+ 152us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 16:58:45.258648 (+445891us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 16:58:45.258826 (+ 178us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46316: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":50}
W20260619 16:58:45.259057 26896 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46316: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260619 16:58:45.259163 26922 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:44.810055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.0.0.1:46316)
0619 16:58:44.810248 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:44.810256 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:44.810310 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:44.810456 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:44.810456 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:44.810458 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0619 16:58:44.810504 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:44.810505 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:44.811522 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:44.811523 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:44.812121 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:44.812124 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:44.812613 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:44.812614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:44.812655 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:44.812661 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 16:58:45.258814 (+446153us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 16:58:45.259014 (+ 200us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.20.231.190:41493: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":95,"threads_started":1}
I20260619 16:58:45.259598 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 26826
2026-06-19T16:58:45Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (624 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (88 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (93 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:58:45 dist-test-slave-vnbj krb5kdc[26934](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:58:45 dist-test-slave-vnbj krb5kdc[26934](info): set up 2 sockets
Jun 19 16:58:45 dist-test-slave-vnbj krb5kdc[26934](info): commencing operation
krb5kdc: starting...
W20260619 16:58:46.543254 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 16:58:46 dist-test-slave-vnbj krb5kdc[26934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:58:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:58:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.231.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
I20260619 16:58:46.665925 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:34151
--webserver_interface=127.20.231.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:35049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:34151
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:46.747452 26947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:46.747673 26947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:46.747697 26947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:46.748900 26947 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:46.748946 26947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:46.748960 26947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:46.748973 26947 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:46.749028 26947 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:58:46.750590 26947 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:35049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:34151
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:34151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:46.750876 26947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:46.751116 26947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:46.756541 26953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:46.756528 26952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:46.756593 26947 server_base.cc:1034] running on GCE node
W20260619 16:58:46.756570 26955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:46.757081 26947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:46.757391 26947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:46.758569 26947 hybrid_clock.cc:648] HybridClock initialized: now 1781888326758552 us; error 34 us; skew 500 ppm
Jun 19 16:58:46 dist-test-slave-vnbj krb5kdc[26934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888326, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:46.761325 26947 init.cc:377] Logged in from keytab as kudu/127.20.231.190@KRBTEST.COM (short username kudu)
I20260619 16:58:46.761765 26947 webserver.cc:460] Webserver started at http://127.20.231.190:43675/ using document root <none> and password file <none>
I20260619 16:58:46.761997 26947 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:46.762066 26947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:46.762219 26947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:46.763379 26947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "ecaaece99b45449bb43a3ae179a8f109"
format_stamp: "Formatted at 2026-06-19 16:58:46 on dist-test-slave-vnbj"
server_key: "be1fd41d43378e69ce13af6451cd46b0"
server_key_iv: "ca7265ae34e261828b5580aaf47541ef"
server_key_version: "encryptionkey@0"
I20260619 16:58:46.763736 26947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "ecaaece99b45449bb43a3ae179a8f109"
format_stamp: "Formatted at 2026-06-19 16:58:46 on dist-test-slave-vnbj"
server_key: "be1fd41d43378e69ce13af6451cd46b0"
server_key_iv: "ca7265ae34e261828b5580aaf47541ef"
server_key_version: "encryptionkey@0"
I20260619 16:58:46.765084 26947 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260619 16:58:46.765813 26962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:46.765957 26947 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:46.766062 26947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "ecaaece99b45449bb43a3ae179a8f109"
format_stamp: "Formatted at 2026-06-19 16:58:46 on dist-test-slave-vnbj"
server_key: "be1fd41d43378e69ce13af6451cd46b0"
server_key_iv: "ca7265ae34e261828b5580aaf47541ef"
server_key_version: "encryptionkey@0"
I20260619 16:58:46.766136 26947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:46.774952 26947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:46.777288 26947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:46.777467 26947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:46.781406 26947 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:34151
I20260619 16:58:46.781441 27014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:34151 every 8 connection(s)
I20260619 16:58:46.781793 26947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 16:58:46.782709 27015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:46.785038 27015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecaaece99b45449bb43a3ae179a8f109: Bootstrap starting.
I20260619 16:58:46.785640 27015 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ecaaece99b45449bb43a3ae179a8f109: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:46.785943 27015 log.cc:824] T 00000000000000000000000000000000 P ecaaece99b45449bb43a3ae179a8f109: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:46.786649 27015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ecaaece99b45449bb43a3ae179a8f109: No bootstrap required, opened a new log
I20260619 16:58:46.788269 27015 raft_consensus.cc:348] T 00000000000000000000000000000000 P ecaaece99b45449bb43a3ae179a8f109 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecaaece99b45449bb43a3ae179a8f109" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 34151 } }
I20260619 16:58:46.788446 27015 raft_consensus.cc:374] T 00000000000000000000000000000000 P ecaaece99b45449bb43a3ae179a8f109 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:46.788470 27015 raft_consensus.cc:729] T 00000000000000000000000000000000 P ecaaece99b45449bb43a3ae179a8f109 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ecaaece99b45449bb43a3ae179a8f109, State: Initialized, Role: FOLLOWER
I20260619 16:58:46.788643 27015 consensus_queue.cc:260] T 00000000000000000000000000000000 P ecaaece99b45449bb43a3ae179a8f109 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecaaece99b45449bb43a3ae179a8f109" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 34151 } }
I20260619 16:58:46.788707 27015 raft_consensus.cc:388] T 00000000000000000000000000000000 P ecaaece99b45449bb43a3ae179a8f109 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:58:46.788730 27015 raft_consensus.cc:482] T 00000000000000000000000000000000 P ecaaece99b45449bb43a3ae179a8f109 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:58:46.788805 27015 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ecaaece99b45449bb43a3ae179a8f109 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:46.789386 27015 raft_consensus.cc:504] T 00000000000000000000000000000000 P ecaaece99b45449bb43a3ae179a8f109 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecaaece99b45449bb43a3ae179a8f109" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 34151 } }
I20260619 16:58:46.789505 27015 leader_election.cc:302] T 00000000000000000000000000000000 P ecaaece99b45449bb43a3ae179a8f109 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ecaaece99b45449bb43a3ae179a8f109; no voters:
I20260619 16:58:46.789698 27015 leader_election.cc:288] T 00000000000000000000000000000000 P ecaaece99b45449bb43a3ae179a8f109 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:58:46.789767 27018 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ecaaece99b45449bb43a3ae179a8f109 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:46.789968 27015 sys_catalog.cc:564] T 00000000000000000000000000000000 P ecaaece99b45449bb43a3ae179a8f109 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:46.790128 27018 raft_consensus.cc:686] T 00000000000000000000000000000000 P ecaaece99b45449bb43a3ae179a8f109 [term 1 LEADER]: Becoming Leader. State: Replica: ecaaece99b45449bb43a3ae179a8f109, State: Running, Role: LEADER
I20260619 16:58:46.790223 27018 consensus_queue.cc:237] T 00000000000000000000000000000000 P ecaaece99b45449bb43a3ae179a8f109 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecaaece99b45449bb43a3ae179a8f109" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 34151 } }
I20260619 16:58:46.790607 27020 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecaaece99b45449bb43a3ae179a8f109 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ecaaece99b45449bb43a3ae179a8f109. Latest consensus state: current_term: 1 leader_uuid: "ecaaece99b45449bb43a3ae179a8f109" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecaaece99b45449bb43a3ae179a8f109" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 34151 } } }
I20260619 16:58:46.790616 27019 sys_catalog.cc:455] T 00000000000000000000000000000000 P ecaaece99b45449bb43a3ae179a8f109 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ecaaece99b45449bb43a3ae179a8f109" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ecaaece99b45449bb43a3ae179a8f109" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 34151 } } }
I20260619 16:58:46.790714 27020 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecaaece99b45449bb43a3ae179a8f109 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:46.790717 27019 sys_catalog.cc:458] T 00000000000000000000000000000000 P ecaaece99b45449bb43a3ae179a8f109 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:46.791014 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 26947
I20260619 16:58:46.791111 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 16:58:46.791271 27027 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:58:46.791301 21406 external_mini_cluster.cc:1428] Setting key 9435fe37691da443e439854e7be76c9a
I20260619 16:58:46.791805 27027 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:58:46.793447 27027 catalog_manager.cc:1347] Generated new cluster ID: 86ffa24eca0d4c17bb643b288d75424b
I20260619 16:58:46.793506 27027 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 16:58:46 dist-test-slave-vnbj krb5kdc[26934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888326, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:46.803143 27027 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:58:46.803869 27027 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:58:46.803936 27037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:46.791789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49320 (local address 127.20.231.190:34151)
0619 16:58:46.792078 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:46.792086 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:46.792110 (+ 24us) server_negotiation.cc:407] Connection header received
0619 16:58:46.792207 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:46.792214 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:46.792259 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:46.792400 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:46.793129 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:46.796640 (+ 3511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:46.797438 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:46.797878 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:46.798002 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:46.800490 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:46.800499 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:46.800499 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:46.800510 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:46.802167 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:46.802549 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:46.802553 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:46.802557 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:46.802643 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:46.802912 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:46.802913 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:46.802913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:46.803250 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:46.803427 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:46.803543 (+ 116us) server_negotiation.cc:299] Negotiation successful
0619 16:58:46.803705 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":74,"threads_started":1}
I20260619 16:58:46.817325 27027 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ecaaece99b45449bb43a3ae179a8f109: Generated new TSK 0
I20260619 16:58:46.817574 27027 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.231.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab.
Entry for principal kudu/127.20.231.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab.
I20260619 16:58:46.850796 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.129:0
--local_ip_for_outbound_sockets=127.20.231.129
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:34151
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:35049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:46.932950 27041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:46.933163 27041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:46.933187 27041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:46.934962 27041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:46.935020 27041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:46.935081 27041 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.129
I20260619 16:58:46.937503 27041 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:35049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:34151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27041
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.129
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:46.937781 27041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:46.938043 27041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:46.943367 27046 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:46.943392 27047 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:46.943392 27049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:46.943415 27041 server_base.cc:1034] running on GCE node
I20260619 16:58:46.943912 27041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:46.944191 27041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:46.945420 27041 hybrid_clock.cc:648] HybridClock initialized: now 1781888326945399 us; error 43 us; skew 500 ppm
Jun 19 16:58:46 dist-test-slave-vnbj krb5kdc[26934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888326, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:46.948215 27041 init.cc:377] Logged in from keytab as kudu/127.20.231.129@KRBTEST.COM (short username kudu)
I20260619 16:58:46.948706 27041 webserver.cc:460] Webserver started at http://127.20.231.129:43919/ using document root <none> and password file <none>
I20260619 16:58:46.948944 27041 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:46.948992 27041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:46.949132 27041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:46.950254 27041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-0/data/instance:
uuid: "60f9f9edcc384929b811d2279d57f574"
format_stamp: "Formatted at 2026-06-19 16:58:46 on dist-test-slave-vnbj"
server_key: "cfe49148ce063267d5fc1c1ac433444d"
server_key_iv: "cebb1ddca16574639357f2ea5b7d2f7c"
server_key_version: "encryptionkey@0"
I20260619 16:58:46.950628 27041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance:
uuid: "60f9f9edcc384929b811d2279d57f574"
format_stamp: "Formatted at 2026-06-19 16:58:46 on dist-test-slave-vnbj"
server_key: "cfe49148ce063267d5fc1c1ac433444d"
server_key_iv: "cebb1ddca16574639357f2ea5b7d2f7c"
server_key_version: "encryptionkey@0"
I20260619 16:58:46.952102 27041 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:58:46.952947 27056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:46.953146 27041 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:46.953305 27041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-0/wal
uuid: "60f9f9edcc384929b811d2279d57f574"
format_stamp: "Formatted at 2026-06-19 16:58:46 on dist-test-slave-vnbj"
server_key: "cfe49148ce063267d5fc1c1ac433444d"
server_key_iv: "cebb1ddca16574639357f2ea5b7d2f7c"
server_key_version: "encryptionkey@0"
I20260619 16:58:46.953416 27041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:46.966729 27041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:46.969276 27041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:46.969458 27041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:46.969715 27041 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:58:46.970057 27041 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:46.970109 27041 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:46.970150 27041 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:46.970176 27041 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:46.976200 27041 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.129:41295
I20260619 16:58:46.976244 27169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.129:41295 every 8 connection(s)
I20260619 16:58:46.976572 27041 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
Jun 19 16:58:46 dist-test-slave-vnbj krb5kdc[26934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888326, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:46.985960 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 27041
I20260619 16:58:46.986083 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance
I20260619 16:58:46.986325 21406 external_mini_cluster.cc:1428] Setting key e5cebb62e42c184dffd63630ee196e67
I20260619 16:58:46.986438 27037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:46.977406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:43337 (local address 127.20.231.190:34151)
0619 16:58:46.977484 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:46.977486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:46.977624 (+ 138us) server_negotiation.cc:407] Connection header received
0619 16:58:46.978594 (+ 970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:46.978596 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:46.978612 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:46.978680 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:46.979410 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:46.980001 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:46.981013 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:46.981229 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:46.981298 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:46.983572 (+ 2274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:46.983574 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:46.983574 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:46.983583 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:46.985063 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:46.985439 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:46.985440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:46.985441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:46.985493 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:46.985705 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:46.985705 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:46.985705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:46.985809 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:46.985910 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:46.986234 (+ 324us) server_negotiation.cc:299] Negotiation successful
0619 16:58:46.986334 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 16:58:46.986649 27172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:46.977315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:34151 (local address 127.20.231.129:43337)
0619 16:58:46.977486 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:46.977532 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:46.978467 (+ 935us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:46.978760 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:46.978763 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:46.979084 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:46.979293 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:46.979298 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:46.980108 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:46.980109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:46.980921 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:46.980923 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:46.981316 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:46.981317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:46.981414 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:46.981956 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:46.981965 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:46.983470 (+ 1505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:46.985140 (+ 1670us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:46.985146 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:46.985147 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:46.985371 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:46.985552 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:46.985553 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:46.985553 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:46.985650 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:46.985902 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:46.985910 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:46.986061 (+ 151us) client_negotiation.cc:769] Sending connection context
0619 16:58:46.986196 (+ 135us) client_negotiation.cc:240] Negotiation successful
0619 16:58:46.986355 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":96,"threads_started":1}
I20260619 16:58:46.987304 27170 heartbeater.cc:347] Connected to a master server at 127.20.231.190:34151
I20260619 16:58:46.987439 27170 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:46.987666 27170 heartbeater.cc:511] Master 127.20.231.190:34151 requested a full tablet report, sending...
I20260619 16:58:46.988287 26979 ts_manager.cc:194] Registered new tserver with Master: 60f9f9edcc384929b811d2279d57f574 (127.20.231.129:41295)
I20260619 16:58:46.989102 26979 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.129@KRBTEST.COM'} at 127.20.231.129:43337
WARNING: no policy specified for kudu/127.20.231.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab.
Entry for principal kudu/127.20.231.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab.
I20260619 16:58:47.013185 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.130:0
--local_ip_for_outbound_sockets=127.20.231.130
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:34151
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:35049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:47.096482 27175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:47.096836 27175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:47.096880 27175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:47.098165 27175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:47.098238 27175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:47.098281 27175 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.130
I20260619 16:58:47.099906 27175 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:35049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:34151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.130
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:47.100195 27175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:47.100449 27175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:47.105726 27183 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:47.105768 27181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:47.105795 27180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:47.106143 27175 server_base.cc:1034] running on GCE node
I20260619 16:58:47.106391 27175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:47.106621 27175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:47.107793 27175 hybrid_clock.cc:648] HybridClock initialized: now 1781888327107760 us; error 48 us; skew 500 ppm
Jun 19 16:58:47 dist-test-slave-vnbj krb5kdc[26934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888327, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:47.110663 27175 init.cc:377] Logged in from keytab as kudu/127.20.231.130@KRBTEST.COM (short username kudu)
I20260619 16:58:47.111270 27175 webserver.cc:460] Webserver started at http://127.20.231.130:40327/ using document root <none> and password file <none>
I20260619 16:58:47.111543 27175 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:47.111622 27175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:47.111784 27175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:47.112937 27175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-1/data/instance:
uuid: "e51a100764204137b3fc88ffbbda6973"
format_stamp: "Formatted at 2026-06-19 16:58:47 on dist-test-slave-vnbj"
server_key: "d52e2649c3c39d21598c5c645f405b52"
server_key_iv: "48da3d3ee113fc463ae41bd04ea490cb"
server_key_version: "encryptionkey@0"
I20260619 16:58:47.113341 27175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance:
uuid: "e51a100764204137b3fc88ffbbda6973"
format_stamp: "Formatted at 2026-06-19 16:58:47 on dist-test-slave-vnbj"
server_key: "d52e2649c3c39d21598c5c645f405b52"
server_key_iv: "48da3d3ee113fc463ae41bd04ea490cb"
server_key_version: "encryptionkey@0"
I20260619 16:58:47.114895 27175 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:58:47.115800 27190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:47.115995 27175 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:47.116097 27175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-1/wal
uuid: "e51a100764204137b3fc88ffbbda6973"
format_stamp: "Formatted at 2026-06-19 16:58:47 on dist-test-slave-vnbj"
server_key: "d52e2649c3c39d21598c5c645f405b52"
server_key_iv: "48da3d3ee113fc463ae41bd04ea490cb"
server_key_version: "encryptionkey@0"
I20260619 16:58:47.116189 27175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:47.127534 27175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:47.130227 27175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:47.130429 27175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:47.130699 27175 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:58:47.131151 27175 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:47.131215 27175 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:47.131261 27175 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:47.131312 27175 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:47.138242 27175 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.130:36871
I20260619 16:58:47.138338 27303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.130:36871 every 8 connection(s)
I20260619 16:58:47.138736 27175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
Jun 19 16:58:47 dist-test-slave-vnbj krb5kdc[26934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888327, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:47.148356 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 27175
I20260619 16:58:47.148460 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance
I20260619 16:58:47.148687 21406 external_mini_cluster.cc:1428] Setting key ff040c63e9e9b70b73a6764e756a7178
I20260619 16:58:47.150030 27037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:47.139723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:48391 (local address 127.20.231.190:34151)
0619 16:58:47.139826 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:47.139829 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:47.140072 (+ 243us) server_negotiation.cc:407] Connection header received
0619 16:58:47.141128 (+ 1056us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:47.141130 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:47.141143 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:47.141204 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:47.141931 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:47.142494 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:47.143776 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:47.143993 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:47.144061 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:47.146602 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:47.146604 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:47.146604 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:47.146612 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:47.148190 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:47.148717 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:47.148719 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:47.148719 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:47.148799 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:47.149156 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:47.149157 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:47.149157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:47.149287 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:47.149402 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:47.149761 (+ 359us) server_negotiation.cc:299] Negotiation successful
0619 16:58:47.149873 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 16:58:47.150067 27306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:47.139669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:34151 (local address 127.20.231.130:48391)
0619 16:58:47.139911 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:47.139960 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:47.141014 (+ 1054us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:47.141270 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:47.141274 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:47.141584 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:47.141805 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:47.141810 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:47.142626 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:47.142627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:47.143665 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:47.143668 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:47.144100 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:47.144101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:47.144229 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:47.144904 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:47.144914 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:47.146498 (+ 1584us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:47.148292 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:47.148299 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:47.148302 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:47.148634 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:47.148885 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:47.148886 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:47.148887 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:47.149068 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:47.149391 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:47.149398 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:47.149606 (+ 208us) client_negotiation.cc:769] Sending connection context
0619 16:58:47.149706 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 16:58:47.149880 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":160,"threads_started":1}
I20260619 16:58:47.150637 27304 heartbeater.cc:347] Connected to a master server at 127.20.231.190:34151
I20260619 16:58:47.150767 27304 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:47.151021 27304 heartbeater.cc:511] Master 127.20.231.190:34151 requested a full tablet report, sending...
I20260619 16:58:47.151491 26979 ts_manager.cc:194] Registered new tserver with Master: e51a100764204137b3fc88ffbbda6973 (127.20.231.130:36871)
I20260619 16:58:47.151937 26979 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.130@KRBTEST.COM'} at 127.20.231.130:48391
WARNING: no policy specified for kudu/127.20.231.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab.
Entry for principal kudu/127.20.231.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab.
I20260619 16:58:47.178074 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.131:0
--local_ip_for_outbound_sockets=127.20.231.131
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:34151
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:35049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:47.265552 27309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:47.265808 27309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:47.265836 27309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:47.267112 27309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:47.267163 27309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:47.267191 27309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.131
I20260619 16:58:47.268803 27309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:35049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:34151
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.131
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:47.269081 27309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:47.269375 27309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:47.274751 27317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:47.274742 27315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:47.274999 27309 server_base.cc:1034] running on GCE node
W20260619 16:58:47.274742 27314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:47.275333 27309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:47.275629 27309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:47.276804 27309 hybrid_clock.cc:648] HybridClock initialized: now 1781888327276797 us; error 34 us; skew 500 ppm
Jun 19 16:58:47 dist-test-slave-vnbj krb5kdc[26934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888327, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:47.283313 27309 init.cc:377] Logged in from keytab as kudu/127.20.231.131@KRBTEST.COM (short username kudu)
I20260619 16:58:47.283838 27309 webserver.cc:460] Webserver started at http://127.20.231.131:35003/ using document root <none> and password file <none>
I20260619 16:58:47.284087 27309 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:47.284160 27309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:47.284327 27309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:47.285508 27309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-2/data/instance:
uuid: "e6c8e1ce9aed4e09b6d59f48d729754a"
format_stamp: "Formatted at 2026-06-19 16:58:47 on dist-test-slave-vnbj"
server_key: "7495df24d1740258e0e014405a0af31a"
server_key_iv: "13f8003e579c81aed2368a2abfcdd065"
server_key_version: "encryptionkey@0"
I20260619 16:58:47.285907 27309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance:
uuid: "e6c8e1ce9aed4e09b6d59f48d729754a"
format_stamp: "Formatted at 2026-06-19 16:58:47 on dist-test-slave-vnbj"
server_key: "7495df24d1740258e0e014405a0af31a"
server_key_iv: "13f8003e579c81aed2368a2abfcdd065"
server_key_version: "encryptionkey@0"
I20260619 16:58:47.287596 27309 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:58:47.288445 27324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:47.288625 27309 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:47.288738 27309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-2/wal
uuid: "e6c8e1ce9aed4e09b6d59f48d729754a"
format_stamp: "Formatted at 2026-06-19 16:58:47 on dist-test-slave-vnbj"
server_key: "7495df24d1740258e0e014405a0af31a"
server_key_iv: "13f8003e579c81aed2368a2abfcdd065"
server_key_version: "encryptionkey@0"
I20260619 16:58:47.288827 27309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:47.320580 27309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:47.323377 27309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:47.323576 27309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:47.323860 27309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:58:47.324218 27309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:47.324277 27309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:47.324334 27309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:47.324369 27309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:47.330731 27309 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.131:45659
I20260619 16:58:47.330783 27437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.131:45659 every 8 connection(s)
I20260619 16:58:47.331171 27309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
I20260619 16:58:47.334074 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 27309
I20260619 16:58:47.334187 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance
I20260619 16:58:47.334381 21406 external_mini_cluster.cc:1428] Setting key 5ebff50efb5e2872caca3e6a7020d930
Jun 19 16:58:47 dist-test-slave-vnbj krb5kdc[26934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888327, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:47.341918 27037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:47.332025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:36693 (local address 127.20.231.190:34151)
0619 16:58:47.332102 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:47.332104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:47.332313 (+ 209us) server_negotiation.cc:407] Connection header received
0619 16:58:47.333283 (+ 970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:47.333284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:47.333297 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:47.333371 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:47.334282 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:47.335171 (+ 889us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:47.336272 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:47.336496 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:47.336560 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:47.338964 (+ 2404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:47.338965 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:47.338966 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:47.338973 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:47.340536 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:47.340997 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:47.340997 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:47.340998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:47.341050 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:47.341345 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:47.341345 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:47.341346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:47.341455 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:47.341508 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:47.341778 (+ 270us) server_negotiation.cc:299] Negotiation successful
0619 16:58:47.341845 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 16:58:47.342156 27440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:47.332027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:34151 (local address 127.20.231.131:36693)
0619 16:58:47.332194 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:47.332238 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:47.333173 (+ 935us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:47.333443 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:47.333447 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:47.333814 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:47.334181 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:47.334188 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:47.335292 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:47.335293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:47.336183 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:47.336186 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:47.336599 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:47.336601 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:47.336784 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:47.337304 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:47.337313 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:47.338820 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:47.340649 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:47.340656 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:47.340658 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:47.340913 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:47.341114 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:47.341115 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:47.341115 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:47.341266 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:47.341515 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:47.341520 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:47.341653 (+ 133us) client_negotiation.cc:769] Sending connection context
0619 16:58:47.341758 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 16:58:47.341904 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":89,"threads_started":1}
I20260619 16:58:47.342880 27438 heartbeater.cc:347] Connected to a master server at 127.20.231.190:34151
I20260619 16:58:47.343019 27438 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:47.343241 27438 heartbeater.cc:511] Master 127.20.231.190:34151 requested a full tablet report, sending...
I20260619 16:58:47.343778 26979 ts_manager.cc:194] Registered new tserver with Master: e6c8e1ce9aed4e09b6d59f48d729754a (127.20.231.131:45659)
I20260619 16:58:47.344334 26979 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.131@KRBTEST.COM'} at 127.20.231.131:36693
I20260619 16:58:47.346411 21406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/krb5kdc/test-admin.keytab.
Jun 19 16:58:47 dist-test-slave-vnbj krb5kdc[26934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:47.364195 21406 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260619 16:58:47.366007 27445 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:47.365125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49782 (local address 127.20.231.129:41295)
0619 16:58:47.365311 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:47.365317 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:47.365331 (+ 14us) server_negotiation.cc:407] Connection header received
0619 16:58:47.365360 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:47.365361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:47.365377 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:47.365448 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:47.365891 (+ 443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49782: BlockingRecv error: recv got EOF from 127.0.0.1:49782 (error 108)
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":97,"threads_started":1}
W20260619 16:58:47.366065 21406 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/krb5kdc/krb5cc' of type FILE
Jun 19 16:58:47 dist-test-slave-vnbj krb5kdc[26934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:47.367136 21406 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 16:58:47 dist-test-slave-vnbj krb5kdc[26934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:47.368149 21406 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 16:58:47 dist-test-slave-vnbj krb5kdc[26934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.129@KRBTEST.COM
I20260619 16:58:47.376398 27445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:47.368746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49798 (local address 127.20.231.129:41295)
0619 16:58:47.368817 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:47.368819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:47.368871 (+ 52us) server_negotiation.cc:407] Connection header received
0619 16:58:47.368953 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:47.368954 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:47.368965 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:47.369035 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:47.369457 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:47.370301 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:47.371100 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:47.371458 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:47.371544 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:47.373727 (+ 2183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:47.373735 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:47.373735 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:47.373742 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:47.375344 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:47.375687 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:47.375689 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:47.375690 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:47.375742 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:47.375937 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:47.375938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:47.375938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:47.376068 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:47.376176 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:47.376258 (+ 82us) server_negotiation.cc:299] Negotiation successful
0619 16:58:47.376321 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260619 16:58:47.378377 27445 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:47.377725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49814 (local address 127.20.231.129:41295)
0619 16:58:47.377807 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:47.377809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:47.377863 (+ 54us) server_negotiation.cc:407] Connection header received
0619 16:58:47.377911 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:47.377912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:47.377922 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:47.377962 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:47.378309 (+ 347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49814: BlockingRecv error: recv got EOF from 127.0.0.1:49814 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260619 16:58:47.378386 21406 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/krb5kdc/krb5cc' of type FILE
Jun 19 16:58:47 dist-test-slave-vnbj krb5kdc[26934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:47.379462 21406 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 16:58:47 dist-test-slave-vnbj krb5kdc[26934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:47.380311 21406 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 16:58:47 dist-test-slave-vnbj krb5kdc[26934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.129@KRBTEST.COM
I20260619 16:58:47.387589 27445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:47.380838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49822 (local address 127.20.231.129:41295)
0619 16:58:47.380906 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:47.380909 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:47.380976 (+ 67us) server_negotiation.cc:407] Connection header received
0619 16:58:47.381016 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:47.381017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:47.381025 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:47.381079 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:47.381393 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:47.381924 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:47.382602 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:47.382812 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:47.382945 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:47.384953 (+ 2008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:47.384955 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:47.384955 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:47.384962 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:47.386499 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:47.386797 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:47.386798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:47.386799 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:47.386908 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:47.387077 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:47.387077 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:47.387078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:47.387175 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:47.387251 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:47.387355 (+ 104us) server_negotiation.cc:299] Negotiation successful
0619 16:58:47.387444 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
W20260619 16:58:47.389113 27445 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:47.388632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49834 (local address 127.20.231.129:41295)
0619 16:58:47.388669 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:47.388670 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:47.388709 (+ 39us) server_negotiation.cc:407] Connection header received
0619 16:58:47.388747 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:47.388748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:47.388756 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:47.388806 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:47.389060 (+ 254us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49834: BlockingRecv error: recv got EOF from 127.0.0.1:49834 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260619 16:58:47.389212 21406 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781888301345144-21406-0/krb5kdc/krb5cc' of type FILE
Jun 19 16:58:47 dist-test-slave-vnbj krb5kdc[26934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:47.390269 21406 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 16:58:47 dist-test-slave-vnbj krb5kdc[26934](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:47.391304 21406 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 16:58:47 dist-test-slave-vnbj krb5kdc[26934](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888327, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.129@KRBTEST.COM
I20260619 16:58:47.399236 27445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:47.391874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49838 (local address 127.20.231.129:41295)
0619 16:58:47.391918 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:47.391920 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:47.391980 (+ 60us) server_negotiation.cc:407] Connection header received
0619 16:58:47.392032 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:47.392033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:47.392040 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:47.392090 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:47.392493 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:47.393021 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:47.393894 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:47.394111 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:47.394183 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:47.396457 (+ 2274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:47.396464 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:47.396464 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:47.396472 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:47.397986 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:47.398373 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:47.398375 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:47.398375 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:47.398440 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:47.398651 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:47.398653 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:47.398653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:47.398767 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:47.398912 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:47.399032 (+ 120us) server_negotiation.cc:299] Negotiation successful
0619 16:58:47.399113 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 16:58:47.399928 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 27041
I20260619 16:58:47.406189 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 27175
I20260619 16:58:47.411350 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 27309
I20260619 16:58:47.416069 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 26947
2026-06-19T16:58:47Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1976 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:58:47 dist-test-slave-vnbj krb5kdc[27458](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:58:47 dist-test-slave-vnbj krb5kdc[27458](info): set up 2 sockets
Jun 19 16:58:47 dist-test-slave-vnbj krb5kdc[27458](info): commencing operation
krb5kdc: starting...
W20260619 16:58:48.455053 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 16:58:48 dist-test-slave-vnbj krb5kdc[27458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:58:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:58:48Z Disabled control of system clock
WARNING: no policy specified for oryx/127.20.231.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.231.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.231.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.190.keytab.
Entry for principal oryx/127.20.231.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.190.keytab.
I20260619 16:58:48.583389 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:36495
--webserver_interface=127.20.231.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.190.keytab
--principal=oryx/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:33295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:36495
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:48.664562 27471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:48.664827 27471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:48.664865 27471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:48.666026 27471 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:48.666098 27471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:48.666122 27471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:48.666142 27471 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:48.666168 27471 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:58:48.667729 27471 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:33295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:36495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.190.keytab
--principal=oryx/127.20.231.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:36495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27471
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:48.668027 27471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:48.668256 27471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:48.673362 27476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:48.673453 27471 server_base.cc:1034] running on GCE node
W20260619 16:58:48.673372 27477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:48.673372 27479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:48.674057 27471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:48.674378 27471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:48.675545 27471 hybrid_clock.cc:648] HybridClock initialized: now 1781888328675538 us; error 40 us; skew 500 ppm
Jun 19 16:58:48 dist-test-slave-vnbj krb5kdc[27458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888328, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:48.678455 27471 init.cc:377] Logged in from keytab as oryx/127.20.231.190@KRBTEST.COM (short username oryx)
I20260619 16:58:48.679024 27471 webserver.cc:460] Webserver started at http://127.20.231.190:45665/ using document root <none> and password file <none>
I20260619 16:58:48.679291 27471 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:48.679363 27471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:48.679525 27471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:48.680622 27471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "a91d2c9bdf114d3bb4390d2709b4cb73"
format_stamp: "Formatted at 2026-06-19 16:58:48 on dist-test-slave-vnbj"
server_key: "94cf9d16e374d4a3a61805eba733e2e8"
server_key_iv: "123b62fd104002456be0ea73e820c088"
server_key_version: "encryptionkey@0"
I20260619 16:58:48.681010 27471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "a91d2c9bdf114d3bb4390d2709b4cb73"
format_stamp: "Formatted at 2026-06-19 16:58:48 on dist-test-slave-vnbj"
server_key: "94cf9d16e374d4a3a61805eba733e2e8"
server_key_iv: "123b62fd104002456be0ea73e820c088"
server_key_version: "encryptionkey@0"
I20260619 16:58:48.682603 27471 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:58:48.683511 27486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:48.683720 27471 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 16:58:48.683822 27471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "a91d2c9bdf114d3bb4390d2709b4cb73"
format_stamp: "Formatted at 2026-06-19 16:58:48 on dist-test-slave-vnbj"
server_key: "94cf9d16e374d4a3a61805eba733e2e8"
server_key_iv: "123b62fd104002456be0ea73e820c088"
server_key_version: "encryptionkey@0"
I20260619 16:58:48.683915 27471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:48.697127 27471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:48.699337 27471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:48.699510 27471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:48.703933 27471 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:36495
I20260619 16:58:48.703985 27538 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:36495 every 8 connection(s)
I20260619 16:58:48.704375 27471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 16:58:48.705353 27539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:48.707835 27539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73: Bootstrap starting.
I20260619 16:58:48.708434 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 27471
I20260619 16:58:48.708542 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 16:58:48.708652 27539 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:48.708748 21406 external_mini_cluster.cc:1428] Setting key bee5b73cc95efe898c322fc18d19c8c2
I20260619 16:58:48.708992 27539 log.cc:824] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:48.709718 27539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73: No bootstrap required, opened a new log
I20260619 16:58:48.711695 27539 raft_consensus.cc:348] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } }
I20260619 16:58:48.711831 27539 raft_consensus.cc:374] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:48.711853 27539 raft_consensus.cc:729] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a91d2c9bdf114d3bb4390d2709b4cb73, State: Initialized, Role: FOLLOWER
I20260619 16:58:48.711988 27539 consensus_queue.cc:260] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } }
I20260619 16:58:48.712080 27539 raft_consensus.cc:388] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:58:48.712105 27539 raft_consensus.cc:482] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:58:48.712180 27539 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:48.712714 27539 raft_consensus.cc:504] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } }
I20260619 16:58:48.712833 27539 leader_election.cc:302] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a91d2c9bdf114d3bb4390d2709b4cb73; no voters:
I20260619 16:58:48.713006 27539 leader_election.cc:288] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:58:48.713039 27544 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:48.713181 27544 raft_consensus.cc:686] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [term 1 LEADER]: Becoming Leader. State: Replica: a91d2c9bdf114d3bb4390d2709b4cb73, State: Running, Role: LEADER
I20260619 16:58:48.713267 27539 sys_catalog.cc:564] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:48.713266 27544 consensus_queue.cc:237] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } }
I20260619 16:58:48.713667 27544 sys_catalog.cc:455] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a91d2c9bdf114d3bb4390d2709b4cb73. Latest consensus state: current_term: 1 leader_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } }
I20260619 16:58:48.713744 27544 sys_catalog.cc:458] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:48.714125 27546 sys_catalog.cc:455] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } }
I20260619 16:58:48.714268 27546 sys_catalog.cc:458] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:48.715004 27548 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:58:48.715528 27548 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 16:58:48 dist-test-slave-vnbj krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM
I20260619 16:58:48.717327 27548 catalog_manager.cc:1347] Generated new cluster ID: 1ee2332bf6294effb4cd869600b75170
I20260619 16:58:48.717391 27548 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:58:48.720119 27543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:48.709304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42598 (local address 127.20.231.190:36495)
0619 16:58:48.709500 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:48.709510 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:48.709547 (+ 37us) server_negotiation.cc:407] Connection header received
0619 16:58:48.710056 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:48.710064 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:48.710123 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:48.710274 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:48.710951 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:48.711818 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:48.712697 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:48.713348 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:48.713771 (+ 423us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:48.716351 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:48.716355 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:48.716355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:48.716368 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:48.718378 (+ 2010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:48.718831 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:48.718834 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:48.718835 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:48.718961 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:48.719184 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:48.719185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:48.719185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:48.719474 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:48.719630 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:48.719763 (+ 133us) server_negotiation.cc:299] Negotiation successful
0619 16:58:48.719897 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":106,"threads_started":1}
I20260619 16:58:48.737633 27548 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:58:48.738425 27548 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:58:48.745573 27548 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73: Generated new TSK 0
I20260619 16:58:48.745803 27548 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.20.231.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.231.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.231.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.129.keytab.
Entry for principal oryx/127.20.231.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.129.keytab.
I20260619 16:58:48.778522 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.129:0
--local_ip_for_outbound_sockets=127.20.231.129
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:36495
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.129.keytab
--principal=oryx/127.20.231.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:33295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:48.861279 27565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:48.861554 27565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:48.861593 27565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:48.862816 27565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:48.862923 27565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:48.862973 27565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.129
I20260619 16:58:48.864578 27565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:33295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.129.keytab
--principal=oryx/127.20.231.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:36495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.129
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:48.864880 27565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:48.865125 27565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:48.870519 27573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:48.870519 27571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:48.870539 27570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:48.870841 27565 server_base.cc:1034] running on GCE node
I20260619 16:58:48.871055 27565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:48.871260 27565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:48.872433 27565 hybrid_clock.cc:648] HybridClock initialized: now 1781888328872419 us; error 32 us; skew 500 ppm
Jun 19 16:58:48 dist-test-slave-vnbj krb5kdc[27458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888328, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:48.875027 27565 init.cc:377] Logged in from keytab as oryx/127.20.231.129@KRBTEST.COM (short username oryx)
I20260619 16:58:48.875517 27565 webserver.cc:460] Webserver started at http://127.20.231.129:38011/ using document root <none> and password file <none>
I20260619 16:58:48.875756 27565 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:48.875837 27565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:48.875989 27565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:48.877075 27565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-0/data/instance:
uuid: "46441ec526ae4a0694fe9f7907de8b8e"
format_stamp: "Formatted at 2026-06-19 16:58:48 on dist-test-slave-vnbj"
server_key: "6a06c6401df2dceb12c449f9367a8848"
server_key_iv: "a1c2968c44831fdb894a5ebd7c7ba50b"
server_key_version: "encryptionkey@0"
I20260619 16:58:48.877435 27565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance:
uuid: "46441ec526ae4a0694fe9f7907de8b8e"
format_stamp: "Formatted at 2026-06-19 16:58:48 on dist-test-slave-vnbj"
server_key: "6a06c6401df2dceb12c449f9367a8848"
server_key_iv: "a1c2968c44831fdb894a5ebd7c7ba50b"
server_key_version: "encryptionkey@0"
I20260619 16:58:48.878887 27565 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:58:48.879710 27580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:48.879943 27565 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:48.880043 27565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-0/wal
uuid: "46441ec526ae4a0694fe9f7907de8b8e"
format_stamp: "Formatted at 2026-06-19 16:58:48 on dist-test-slave-vnbj"
server_key: "6a06c6401df2dceb12c449f9367a8848"
server_key_iv: "a1c2968c44831fdb894a5ebd7c7ba50b"
server_key_version: "encryptionkey@0"
I20260619 16:58:48.880128 27565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:48.891595 27565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:48.893760 27565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:48.893945 27565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:48.894186 27565 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:58:48.894495 27565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:48.894551 27565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:48.894593 27565 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:48.894618 27565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:48.901475 27565 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.129:36425
I20260619 16:58:48.901571 27693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.129:36425 every 8 connection(s)
I20260619 16:58:48.901914 27565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
I20260619 16:58:48.904073 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 27565
I20260619 16:58:48.904155 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance
I20260619 16:58:48.904342 21406 external_mini_cluster.cc:1428] Setting key 402cec6a37d8f6c138ee63d31c50a262
Jun 19 16:58:48 dist-test-slave-vnbj krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888328, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM
I20260619 16:58:48.912642 27543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:48.902723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:35603 (local address 127.20.231.190:36495)
0619 16:58:48.902769 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:48.902771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:48.903041 (+ 270us) server_negotiation.cc:407] Connection header received
0619 16:58:48.903546 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:48.903547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:48.903557 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:48.903610 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:48.904772 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:48.905382 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:48.906832 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:48.907107 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:48.907165 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:48.909585 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:48.909587 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:48.909588 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:48.909597 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:48.911250 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:48.911642 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:48.911643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:48.911644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:48.911709 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:48.911964 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:48.911965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:48.911965 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:48.912119 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:48.912189 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:48.912436 (+ 247us) server_negotiation.cc:299] Negotiation successful
0619 16:58:48.912525 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 16:58:48.912716 27696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:48.902700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36495 (local address 127.20.231.129:35603)
0619 16:58:48.902915 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:48.902967 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:48.903453 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:48.903663 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:48.903666 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:48.904309 (+ 643us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:48.904628 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:48.904636 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:48.905477 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:48.905478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:48.906706 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:48.906710 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:48.907196 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:48.907198 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:48.907356 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:48.908034 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:48.908043 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:48.909489 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:48.911342 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:48.911348 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:48.911349 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:48.911567 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:48.911778 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:48.911779 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:48.911779 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:48.911895 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:48.912183 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:48.912187 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:48.912303 (+ 116us) client_negotiation.cc:769] Sending connection context
0619 16:58:48.912386 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 16:58:48.912506 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":102,"threads_started":1}
I20260619 16:58:48.913235 27694 heartbeater.cc:347] Connected to a master server at 127.20.231.190:36495
WARNING: no policy specified for oryx/127.20.231.130@KRBTEST.COM; defaulting to no policy
I20260619 16:58:48.913338 27694 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:48.913602 27694 heartbeater.cc:511] Master 127.20.231.190:36495 requested a full tablet report, sending...
I20260619 16:58:48.914186 27503 ts_manager.cc:194] Registered new tserver with Master: 46441ec526ae4a0694fe9f7907de8b8e (127.20.231.129:36425)
I20260619 16:58:48.914925 27503 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.231.129@KRBTEST.COM'} at 127.20.231.129:35603
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.231.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.231.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.130.keytab.
Entry for principal oryx/127.20.231.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.130.keytab.
I20260619 16:58:48.930294 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.130:0
--local_ip_for_outbound_sockets=127.20.231.130
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:36495
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.130.keytab
--principal=oryx/127.20.231.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:33295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:49.012595 27699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:49.012913 27699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:49.012953 27699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:49.014187 27699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:49.014254 27699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:49.014305 27699 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.130
I20260619 16:58:49.015983 27699 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:33295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.130.keytab
--principal=oryx/127.20.231.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:36495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.130
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:49.016253 27699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:49.016500 27699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:49.021944 27707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:49.021934 27705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:49.021914 27699 server_base.cc:1034] running on GCE node
W20260619 16:58:49.021905 27704 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:49.022346 27699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:49.022544 27699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:49.023677 27699 hybrid_clock.cc:648] HybridClock initialized: now 1781888329023663 us; error 30 us; skew 500 ppm
Jun 19 16:58:49 dist-test-slave-vnbj krb5kdc[27458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888329, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:49.026247 27699 init.cc:377] Logged in from keytab as oryx/127.20.231.130@KRBTEST.COM (short username oryx)
I20260619 16:58:49.026672 27699 webserver.cc:460] Webserver started at http://127.20.231.130:42259/ using document root <none> and password file <none>
I20260619 16:58:49.026962 27699 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:49.027011 27699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:49.027163 27699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:49.028247 27699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-1/data/instance:
uuid: "3f0c706426784b9189a3f41b0340d499"
format_stamp: "Formatted at 2026-06-19 16:58:49 on dist-test-slave-vnbj"
server_key: "4d97b7c744a22990f69cd963f6b44e3b"
server_key_iv: "7604fe0cdf218895dc154952c2b68b66"
server_key_version: "encryptionkey@0"
I20260619 16:58:49.028617 27699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance:
uuid: "3f0c706426784b9189a3f41b0340d499"
format_stamp: "Formatted at 2026-06-19 16:58:49 on dist-test-slave-vnbj"
server_key: "4d97b7c744a22990f69cd963f6b44e3b"
server_key_iv: "7604fe0cdf218895dc154952c2b68b66"
server_key_version: "encryptionkey@0"
I20260619 16:58:49.030048 27699 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:58:49.030992 27714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:49.031193 27699 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:58:49.031291 27699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-1/wal
uuid: "3f0c706426784b9189a3f41b0340d499"
format_stamp: "Formatted at 2026-06-19 16:58:49 on dist-test-slave-vnbj"
server_key: "4d97b7c744a22990f69cd963f6b44e3b"
server_key_iv: "7604fe0cdf218895dc154952c2b68b66"
server_key_version: "encryptionkey@0"
I20260619 16:58:49.031348 27699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:49.046725 27699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:49.048898 27699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:49.049091 27699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:49.049400 27699 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:58:49.049764 27699 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:49.049803 27699 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:49.049829 27699 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:49.049844 27699 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:49.056571 27699 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.130:34667
I20260619 16:58:49.056613 27827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.130:34667 every 8 connection(s)
I20260619 16:58:49.056994 27699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
Jun 19 16:58:49 dist-test-slave-vnbj krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888329, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM
I20260619 16:58:49.065466 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 27699
I20260619 16:58:49.065548 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance
I20260619 16:58:49.065752 21406 external_mini_cluster.cc:1428] Setting key 67bd9ded6e8803badcb6f349dc9e6411
I20260619 16:58:49.067215 27543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.057867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:45401 (local address 127.20.231.190:36495)
0619 16:58:49.057952 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:49.057955 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:49.058121 (+ 166us) server_negotiation.cc:407] Connection header received
0619 16:58:49.058629 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:49.058630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:49.058641 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:49.058710 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:49.059761 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.060303 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.061274 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.061480 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.061546 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.064067 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:49.064069 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:49.064069 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:49.064076 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:49.065508 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.066036 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.066038 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.066039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.066108 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.066380 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.066380 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.066381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.066563 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:49.066704 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:49.067031 (+ 327us) server_negotiation.cc:299] Negotiation successful
0619 16:58:49.067116 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 16:58:49.067440 27830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.057812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36495 (local address 127.20.231.130:45401)
0619 16:58:49.057996 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:49.058033 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:49.058510 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:49.058798 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:49.058802 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:49.059438 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:49.059641 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.059646 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.060394 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.060394 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.061183 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.061185 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.061562 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.061562 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.061667 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.062268 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:49.062277 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:49.063936 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:49.065636 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.065643 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.065646 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.065955 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.066180 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.066181 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.066181 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.066310 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.066662 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:49.066668 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:49.066831 (+ 163us) client_negotiation.cc:769] Sending connection context
0619 16:58:49.067007 (+ 176us) client_negotiation.cc:240] Negotiation successful
0619 16:58:49.067191 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":100,"threads_started":1}
I20260619 16:58:49.067890 27828 heartbeater.cc:347] Connected to a master server at 127.20.231.190:36495
I20260619 16:58:49.068002 27828 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:49.068218 27828 heartbeater.cc:511] Master 127.20.231.190:36495 requested a full tablet report, sending...
I20260619 16:58:49.068619 27503 ts_manager.cc:194] Registered new tserver with Master: 3f0c706426784b9189a3f41b0340d499 (127.20.231.130:34667)
I20260619 16:58:49.069151 27503 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.231.130@KRBTEST.COM'} at 127.20.231.130:45401
WARNING: no policy specified for oryx/127.20.231.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.231.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.231.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.131.keytab.
Entry for principal oryx/127.20.231.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.131.keytab.
I20260619 16:58:49.093663 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.131:0
--local_ip_for_outbound_sockets=127.20.231.131
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:36495
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.131.keytab
--principal=oryx/127.20.231.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:33295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:49.177978 27833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:49.178229 27833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:49.178256 27833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:49.179464 27833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:49.179508 27833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:49.179535 27833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.131
I20260619 16:58:49.180991 27833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:33295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.131.keytab
--principal=oryx/127.20.231.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:36495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27833
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.131
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:49.181192 27833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:49.181373 27833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:49.187054 27838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:49.187050 27841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:49.187054 27839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:49.187403 27833 server_base.cc:1034] running on GCE node
I20260619 16:58:49.187639 27833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:49.187902 27833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:49.189154 27833 hybrid_clock.cc:648] HybridClock initialized: now 1781888329189124 us; error 44 us; skew 500 ppm
Jun 19 16:58:49 dist-test-slave-vnbj krb5kdc[27458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888329, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:49.191870 27833 init.cc:377] Logged in from keytab as oryx/127.20.231.131@KRBTEST.COM (short username oryx)
I20260619 16:58:49.192400 27833 webserver.cc:460] Webserver started at http://127.20.231.131:33695/ using document root <none> and password file <none>
I20260619 16:58:49.192646 27833 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:49.192716 27833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:49.192876 27833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:49.193986 27833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-2/data/instance:
uuid: "258ee1befa62410987844618763d8c16"
format_stamp: "Formatted at 2026-06-19 16:58:49 on dist-test-slave-vnbj"
server_key: "50ad76ab62f80a7163ac2735e83a0f93"
server_key_iv: "72beb9ce2bd5cc8a191b0c5b2cbef1fa"
server_key_version: "encryptionkey@0"
I20260619 16:58:49.194355 27833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance:
uuid: "258ee1befa62410987844618763d8c16"
format_stamp: "Formatted at 2026-06-19 16:58:49 on dist-test-slave-vnbj"
server_key: "50ad76ab62f80a7163ac2735e83a0f93"
server_key_iv: "72beb9ce2bd5cc8a191b0c5b2cbef1fa"
server_key_version: "encryptionkey@0"
I20260619 16:58:49.195952 27833 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:58:49.196807 27848 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:49.196998 27833 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 16:58:49.197086 27833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-2/wal
uuid: "258ee1befa62410987844618763d8c16"
format_stamp: "Formatted at 2026-06-19 16:58:49 on dist-test-slave-vnbj"
server_key: "50ad76ab62f80a7163ac2735e83a0f93"
server_key_iv: "72beb9ce2bd5cc8a191b0c5b2cbef1fa"
server_key_version: "encryptionkey@0"
I20260619 16:58:49.197166 27833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:49.217007 27833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:49.219394 27833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:49.219570 27833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:49.219821 27833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:58:49.220132 27833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:49.220187 27833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:49.220232 27833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:49.220256 27833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:49.226293 27833 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.131:36771
I20260619 16:58:49.226341 27961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.131:36771 every 8 connection(s)
I20260619 16:58:49.226666 27833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
I20260619 16:58:49.228837 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 27833
I20260619 16:58:49.228945 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance
I20260619 16:58:49.229108 21406 external_mini_cluster.cc:1428] Setting key 7a875c8148d2205b49860d1fc21025b9
Jun 19 16:58:49 dist-test-slave-vnbj krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888329, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM
I20260619 16:58:49.236804 27543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.227439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:35699 (local address 127.20.231.190:36495)
0619 16:58:49.227500 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:49.227502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:49.227723 (+ 221us) server_negotiation.cc:407] Connection header received
0619 16:58:49.228323 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:49.228324 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:49.228337 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:49.228388 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:49.229430 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.229938 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.231055 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.231261 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.231348 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.233710 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:49.233712 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:49.233713 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:49.233720 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:49.235341 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.235789 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.235791 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.235792 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.235857 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.236128 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.236129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.236129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.236250 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:49.236337 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:49.236622 (+ 285us) server_negotiation.cc:299] Negotiation successful
0619 16:58:49.236708 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 16:58:49.237011 27964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.227424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36495 (local address 127.20.231.131:35699)
0619 16:58:49.227604 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:49.227640 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:49.228184 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:49.228462 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:49.228465 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:49.229087 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:49.229325 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.229330 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.230035 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.230036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.230959 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.230961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.231358 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.231359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.231497 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.232077 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:49.232085 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:49.233606 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:49.235439 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.235447 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.235449 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.235701 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.235930 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.235931 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.235931 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.236051 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.236325 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:49.236331 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:49.236485 (+ 154us) client_negotiation.cc:769] Sending connection context
0619 16:58:49.236591 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 16:58:49.236758 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":97,"threads_started":1}
I20260619 16:58:49.237564 27962 heartbeater.cc:347] Connected to a master server at 127.20.231.190:36495
I20260619 16:58:49.237650 27962 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:49.237844 27962 heartbeater.cc:511] Master 127.20.231.190:36495 requested a full tablet report, sending...
I20260619 16:58:49.238266 27503 ts_manager.cc:194] Registered new tserver with Master: 258ee1befa62410987844618763d8c16 (127.20.231.131:36771)
I20260619 16:58:49.238641 27503 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.231.131@KRBTEST.COM'} at 127.20.231.131:35699
I20260619 16:58:49.240978 21406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.20.231.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.231.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.231.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.189.keytab.
Entry for principal oryx/127.20.231.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.189.keytab.
I20260619 16:58:49.266505 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.189:44821
--webserver_interface=127.20.231.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.189.keytab
--principal=oryx/127.20.231.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:33295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:36495,127.20.231.189:44821
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:49.349964 27967 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:49.350188 27967 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:49.350211 27967 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:49.351518 27967 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:49.351563 27967 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:49.351578 27967 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:49.351591 27967 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:49.351604 27967 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:58:49.353080 27967 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:33295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:36495,127.20.231.189:44821
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.189.keytab
--principal=oryx/127.20.231.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.189:44821
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.231.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27967
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
W20260619 16:58:49.353189 27967 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.20.231.190:36495,127.20.231.189:44821'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260619 16:58:49.353371 27967 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:49.353600 27967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:49.358981 27975 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:49.359014 27973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:49.359014 27967 server_base.cc:1034] running on GCE node
W20260619 16:58:49.358981 27972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:49.359570 27967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:49.359961 27967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:49.361208 27967 hybrid_clock.cc:648] HybridClock initialized: now 1781888329361185 us; error 34 us; skew 500 ppm
Jun 19 16:58:49 dist-test-slave-vnbj krb5kdc[27458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888329, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:49.364060 27967 init.cc:377] Logged in from keytab as oryx/127.20.231.189@KRBTEST.COM (short username oryx)
I20260619 16:58:49.364460 27967 webserver.cc:460] Webserver started at http://127.20.231.189:41731/ using document root <none> and password file <none>
I20260619 16:58:49.364663 27967 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:49.364722 27967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:49.364815 27967 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:49.365828 27967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-1/data/instance:
uuid: "674b2e8dfe9a4a8d82791cd75c0e0606"
format_stamp: "Formatted at 2026-06-19 16:58:49 on dist-test-slave-vnbj"
server_key: "03166bf2115b3ffac50f40bbe9db200e"
server_key_iv: "e09bd9f9cf77af8cc82a5539bda442de"
server_key_version: "encryptionkey@0"
I20260619 16:58:49.366221 27967 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-1/wal/instance:
uuid: "674b2e8dfe9a4a8d82791cd75c0e0606"
format_stamp: "Formatted at 2026-06-19 16:58:49 on dist-test-slave-vnbj"
server_key: "03166bf2115b3ffac50f40bbe9db200e"
server_key_iv: "e09bd9f9cf77af8cc82a5539bda442de"
server_key_version: "encryptionkey@0"
I20260619 16:58:49.367583 27967 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:58:49.368314 27982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:49.368491 27967 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:58:49.368592 27967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-1/wal
uuid: "674b2e8dfe9a4a8d82791cd75c0e0606"
format_stamp: "Formatted at 2026-06-19 16:58:49 on dist-test-slave-vnbj"
server_key: "03166bf2115b3ffac50f40bbe9db200e"
server_key_iv: "e09bd9f9cf77af8cc82a5539bda442de"
server_key_version: "encryptionkey@0"
I20260619 16:58:49.368640 27967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:49.393232 27967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:49.395725 27967 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:49.395885 27967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:49.399734 27967 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.189:44821
I20260619 16:58:49.399799 28034 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.189:44821 every 8 connection(s)
I20260619 16:58:49.400178 27967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-1/data/info.pb
I20260619 16:58:49.401413 28035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:49.402108 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 27967
I20260619 16:58:49.403522 28035 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 16:58:49 dist-test-slave-vnbj krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888329, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.189@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM
Jun 19 16:58:49 dist-test-slave-vnbj krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.231.189@KRBTEST.COM
I20260619 16:58:49.413054 27543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.404398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42600 (local address 127.20.231.190:36495)
0619 16:58:49.404450 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:49.404452 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:49.404587 (+ 135us) server_negotiation.cc:407] Connection header received
0619 16:58:49.404714 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:49.404715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:49.404728 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:49.404794 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:49.405317 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.405821 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.406761 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.407036 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.407122 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.409383 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:49.409384 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:49.409385 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:49.409391 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:49.411356 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.411897 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.411899 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.411899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.411972 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.412246 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.412248 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.412248 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.412416 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:49.412496 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:49.412847 (+ 351us) server_negotiation.cc:299] Negotiation successful
0619 16:58:49.412955 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 16:58:49.413266 28039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.404296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36495 (local address 127.0.0.1:42600)
0619 16:58:49.404458 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:49.404506 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:49.404607 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:49.404820 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:49.404823 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:49.405117 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:49.405212 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.405217 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.405948 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.405949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.406642 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.406644 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.407129 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.407130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.407212 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.407786 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:49.407794 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:49.409283 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:49.411453 (+ 2170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.411459 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.411460 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.411806 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.412053 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.412054 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.412054 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.412165 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.412496 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:49.412503 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:49.412702 (+ 199us) client_negotiation.cc:769] Sending connection context
0619 16:58:49.412824 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 16:58:49.413031 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":30,"thread_start_us":84,"threads_started":1}
I20260619 16:58:49.413655 28038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.402808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50804 (local address 127.20.231.189:44821)
0619 16:58:49.402997 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:49.403007 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:49.403074 (+ 67us) server_negotiation.cc:407] Connection header received
0619 16:58:49.403629 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:49.403638 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:49.403705 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:49.403888 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:49.404704 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.405594 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.406338 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.406758 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.406897 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.409983 (+ 3086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:49.409991 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:49.409992 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:49.409999 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:49.412056 (+ 2057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.412431 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.412435 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.412435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.412500 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.412752 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.412753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.412753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.412958 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:49.413121 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:49.413454 (+ 333us) server_negotiation.cc:299] Negotiation successful
0619 16:58:49.413530 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":138,"thread_start_us":99,"threads_started":1}
I20260619 16:58:49.414043 28035 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 16:58:49 dist-test-slave-vnbj krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888329, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.189@KRBTEST.COM for oryx/127.20.231.189@KRBTEST.COM
I20260619 16:58:49.420428 28039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.414323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.189:44821 (local address 127.0.0.1:50810)
0619 16:58:49.414377 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:49.414388 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:49.414429 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:49.414624 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:49.414624 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:49.414737 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:49.414770 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.414771 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.415421 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.415421 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.415961 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.415963 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.416306 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.416306 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.416361 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.416777 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:49.416781 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:49.417932 (+ 1151us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:49.419444 (+ 1512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.419444 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.419445 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.419596 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.419799 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.419799 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.419800 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.419849 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.420153 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:49.420154 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:49.420194 (+ 40us) client_negotiation.cc:769] Sending connection context
0619 16:58:49.420284 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 16:58:49.420365 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 16:58:49.420491 28038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.414391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50810 (local address 127.20.231.189:44821)
0619 16:58:49.414484 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:49.414486 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:49.414494 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:58:49.414538 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:49.414539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:49.414550 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:49.414627 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:49.414819 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.415346 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.416037 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.416239 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.416301 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.418006 (+ 1705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:49.418007 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:49.418007 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:49.418013 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:49.419345 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.419685 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.419686 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.419686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.419739 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.419930 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.419931 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.419931 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.420039 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:49.420103 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:49.420314 (+ 211us) server_negotiation.cc:299] Negotiation successful
0619 16:58:49.420412 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":74}
I20260619 16:58:49.422021 28035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606: Bootstrap starting.
I20260619 16:58:49.422957 28035 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:49.423239 28035 log.cc:824] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:49.423997 28035 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606: No bootstrap required, opened a new log
I20260619 16:58:49.425387 28035 raft_consensus.cc:348] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } }
I20260619 16:58:49.425530 28035 raft_consensus.cc:374] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:49.425577 28035 raft_consensus.cc:729] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 674b2e8dfe9a4a8d82791cd75c0e0606, State: Initialized, Role: FOLLOWER
I20260619 16:58:49.425737 28035 consensus_queue.cc:260] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } }
I20260619 16:58:49.426095 28035 sys_catalog.cc:564] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:49.426067 28041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } } }
I20260619 16:58:49.426164 28041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 16:58:49.427922 28052 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 16:58:49.427981 28052 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 16:58:49.429539 27967 master_runner.cc:428] Detected that this master 674b2e8dfe9a4a8d82791cd75c0e0606 is joining an existing cluster
I20260619 16:58:49.429584 27967 master_runner.cc:432] Initiating AddMaster RPC to add 127.20.231.189:44821
I20260619 16:58:49.437345 27543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.430517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42616 (local address 127.20.231.190:36495)
0619 16:58:49.430570 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:49.430572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:49.430673 (+ 101us) server_negotiation.cc:407] Connection header received
0619 16:58:49.430828 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:49.430829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:49.430840 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:49.430887 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:49.431735 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.432349 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.433126 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.433449 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.433519 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.434678 (+ 1159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:49.434679 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:49.434680 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:49.434687 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:49.436159 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.436551 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.436552 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.436553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.436629 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.436816 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.436817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.436817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.436938 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:49.437015 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:49.437166 (+ 151us) server_negotiation.cc:299] Negotiation successful
0619 16:58:49.437247 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 16:58:49.437372 28058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.430449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36495 (local address 127.0.0.1:42616)
0619 16:58:49.430614 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:49.430624 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:49.430749 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:49.430953 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:49.430954 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:49.431582 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:49.431645 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.431647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.432426 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.432427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.433013 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.433015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.433523 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.433523 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.433582 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.434172 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:49.434177 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:49.434579 (+ 402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:49.436247 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.436248 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.436249 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.436463 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.436679 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.436680 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.436680 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.436746 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.437005 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:49.437006 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:49.437051 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 16:58:49.437141 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 16:58:49.437248 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":3,"thread_start_us":83,"threads_started":1}
I20260619 16:58:49.438330 28038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.430708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50812 (local address 127.20.231.189:44821)
0619 16:58:49.430754 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:49.430756 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:49.430955 (+ 199us) server_negotiation.cc:407] Connection header received
0619 16:58:49.431023 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:49.431024 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:49.431033 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:49.431087 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:49.431418 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.432015 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.433114 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.433316 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.433390 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.434638 (+ 1248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:49.434639 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:49.434640 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:49.434647 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:49.436888 (+ 2241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.437256 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.437257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.437257 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.437305 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.437814 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.437815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.437816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.437945 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:49.437986 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:49.438163 (+ 177us) server_negotiation.cc:299] Negotiation successful
0619 16:58:49.438244 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 16:58:49.438917 28059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.430676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.189:44821 (local address 127.0.0.1:50812)
0619 16:58:49.430841 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:49.430915 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:49.430968 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:49.431091 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:49.431092 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:49.431265 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:49.431342 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.431344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.432102 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.432102 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.433004 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.433006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.433414 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.433415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.433477 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.434034 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:49.434039 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:49.434550 (+ 511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:49.436977 (+ 2427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.436977 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.436978 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.437168 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.437370 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.437371 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.437371 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.437737 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.438027 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:49.438028 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:49.438060 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 16:58:49.438437 (+ 377us) client_negotiation.cc:240] Negotiation successful
0619 16:58:49.438789 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":26,"thread_start_us":92,"threads_started":1}
Jun 19 16:58:49 dist-test-slave-vnbj krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888328, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.190@KRBTEST.COM for oryx/127.20.231.189@KRBTEST.COM
I20260619 16:58:49.446823 28038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.439943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50820 (local address 127.20.231.189:44821)
0619 16:58:49.439976 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:49.439978 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:49.440159 (+ 181us) server_negotiation.cc:407] Connection header received
0619 16:58:49.440247 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:49.440248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:49.440258 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:49.440322 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:49.440731 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.441189 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.441999 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.442198 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.442272 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.444324 (+ 2052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:49.444325 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:49.444326 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:49.444333 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:49.445621 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.446016 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.446018 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.446018 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.446075 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.446289 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.446290 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.446290 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.446395 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:49.446463 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:49.446648 (+ 185us) server_negotiation.cc:299] Negotiation successful
0619 16:58:49.446725 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 16:58:49.446897 28060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.439905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.189:44821 (local address 127.0.0.1:50820)
0619 16:58:49.440040 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:49.440095 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:49.440183 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:49.440327 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:49.440331 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:49.440597 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:49.440643 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.440645 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.441270 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.441271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.441903 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.441905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.442271 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.442272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.442353 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.442945 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:49.442950 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:49.444230 (+ 1280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:49.445707 (+ 1477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.445714 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.445717 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.445925 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.446143 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.446144 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.446144 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.446210 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.446468 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:49.446476 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:49.446527 (+ 51us) client_negotiation.cc:769] Sending connection context
0619 16:58:49.446624 (+ 97us) client_negotiation.cc:240] Negotiation successful
0619 16:58:49.446739 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":71,"threads_started":1}
I20260619 16:58:49.447515 27503 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.20.231.189:44821: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606"
member_type: NON_VOTER
last_known_addr {
host: "127.20.231.189"
port: 44821
}
attrs {
promote: true
}
}
dest_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73"
cas_config_opid_index: -1
I20260619 16:58:49.447853 27503 consensus_queue.cc:237] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: NON_VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: true } }
I20260619 16:58:49.448396 28062 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 674b2e8dfe9a4a8d82791cd75c0e0606 (127.20.231.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: NON_VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: true } } }
I20260619 16:58:49.448695 28009 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:49.449424 28009 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [term 1 FOLLOWER]: Refusing update from remote peer a91d2c9bdf114d3bb4390d2709b4cb73: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260619 16:58:49.449581 28062 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.20.231.189:44821
I20260619 16:58:49.449668 28062 sys_catalog.cc:455] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: NON_VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: true } } }
I20260619 16:58:49.449723 28062 sys_catalog.cc:458] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:49.449852 27967 master.cc:537] Master@127.20.231.189:44821 shutting down...
I20260619 16:58:49.449828 28062 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: NON_VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 16:58:49.450522 27488 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260619 16:58:49.451148 27488 consensus_peers.cc:487] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 -> Peer 674b2e8dfe9a4a8d82791cd75c0e0606 (127.20.231.189:44821): Couldn't send request to peer 674b2e8dfe9a4a8d82791cd75c0e0606. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 16:58:49.452559 27967 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:58:49.452661 27967 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:49.452721 27967 tablet_replica.cc:330] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606: stopping tablet replica
I20260619 16:58:49.465251 27967 master.cc:552] Master@127.20.231.189:44821 shutdown complete.
I20260619 16:58:49.465348 27967 master_runner.cc:305] Clearing existing system tablet
I20260619 16:58:49.465473 27967 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:49.465533 27967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:49.466143 27967 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:58:49.466658 28066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:49.466832 27967 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:49.466929 27967 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 16:58:49.466956 27967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-1/wal
uuid: "674b2e8dfe9a4a8d82791cd75c0e0606"
format_stamp: "Formatted at 2026-06-19 16:58:49 on dist-test-slave-vnbj"
server_key: "03166bf2115b3ffac50f40bbe9db200e"
server_key_iv: "e09bd9f9cf77af8cc82a5539bda442de"
server_key_version: "encryptionkey@0"
I20260619 16:58:49.468209 27967 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:49.469743 27967 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 16:58:49.469816 27967 log.cc:1196] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260619 16:58:49.470010 27967 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606: Deleting consensus metadata
I20260619 16:58:49.470157 27967 master_runner.cc:315] Copying system tablet from 127.20.231.190:36495
I20260619 16:58:49.470762 27967 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606: tablet copy: Beginning tablet copy session from remote peer at address 127.20.231.190:36495
I20260619 16:58:49.477810 27543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.471202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42624 (local address 127.20.231.190:36495)
0619 16:58:49.471252 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:49.471255 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:49.471320 (+ 65us) server_negotiation.cc:407] Connection header received
0619 16:58:49.471403 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:49.471404 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:49.471416 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:49.471476 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:49.471747 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.472549 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.473286 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.473590 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.473660 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.474631 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:49.474633 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:49.474633 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:49.474640 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:49.476424 (+ 1784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.476748 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.476749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.476749 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.476810 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.477096 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.477098 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.477098 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.477306 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:49.477395 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:49.477630 (+ 235us) server_negotiation.cc:299] Negotiation successful
0619 16:58:49.477715 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 16:58:49.478091 28072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.471116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36495 (local address 127.0.0.1:42624)
0619 16:58:49.471253 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:49.471263 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:49.471334 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:49.471479 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:49.471480 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:49.471630 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:49.471683 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.471685 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.472637 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.472638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.473204 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.473206 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.473668 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.473669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.473721 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.474251 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:49.474255 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:49.474546 (+ 291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:49.476511 (+ 1965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.476512 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.476513 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.476678 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.476904 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.476905 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.476905 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.477012 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.477416 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:49.477417 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:49.477491 (+ 74us) client_negotiation.cc:769] Sending connection context
0619 16:58:49.477600 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 16:58:49.477753 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":83,"threads_started":1}
I20260619 16:58:49.478541 27523 tablet_copy_service.cc:140] P a91d2c9bdf114d3bb4390d2709b4cb73: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 674b2e8dfe9a4a8d82791cd75c0e0606 ({username='oryx', principal='oryx/127.20.231.189@KRBTEST.COM'} at 127.0.0.1:42624)
I20260619 16:58:49.478649 27523 tablet_copy_service.cc:161] P a91d2c9bdf114d3bb4390d2709b4cb73: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 674b2e8dfe9a4a8d82791cd75c0e0606 at {username='oryx', principal='oryx/127.20.231.189@KRBTEST.COM'} at 127.0.0.1:42624: session id = 674b2e8dfe9a4a8d82791cd75c0e0606-00000000000000000000000000000000
I20260619 16:58:49.479406 27523 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 16:58:49.480088 27967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 16:58:49.481472 27967 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606: tablet copy: Starting download of 0 data blocks...
I20260619 16:58:49.481631 27967 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606: tablet copy: Starting download of 1 WAL segments...
I20260619 16:58:49.482511 27967 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 16:58:49.483529 27523 tablet_copy_service.cc:342] P a91d2c9bdf114d3bb4390d2709b4cb73: Request end of tablet copy session 674b2e8dfe9a4a8d82791cd75c0e0606-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.20.231.189@KRBTEST.COM'} at 127.0.0.1:42624
I20260619 16:58:49.483595 27523 tablet_copy_service.cc:433] P a91d2c9bdf114d3bb4390d2709b4cb73: ending tablet copy session 674b2e8dfe9a4a8d82791cd75c0e0606-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 674b2e8dfe9a4a8d82791cd75c0e0606
I20260619 16:58:49.486374 27967 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:49.487702 28073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:49.487707 28074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:49.487746 28076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:49.487920 27967 server_base.cc:1034] running on GCE node
I20260619 16:58:49.488090 27967 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:49.488237 27967 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:49.489367 27967 hybrid_clock.cc:648] HybridClock initialized: now 1781888329489360 us; error 26 us; skew 500 ppm
Jun 19 16:58:49 dist-test-slave-vnbj krb5kdc[27458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888329, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:49.490919 27967 init.cc:377] Logged in from keytab as oryx/127.20.231.189@KRBTEST.COM (short username oryx)
I20260619 16:58:49.491235 27967 webserver.cc:460] Webserver started at http://127.20.231.189:40471/ using document root <none> and password file <none>
I20260619 16:58:49.491358 27967 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:49.491398 27967 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:49.492012 27967 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:58:49.492537 28083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:49.492702 27967 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:58:49.492761 27967 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-1/wal
uuid: "674b2e8dfe9a4a8d82791cd75c0e0606"
format_stamp: "Formatted at 2026-06-19 16:58:49 on dist-test-slave-vnbj"
server_key: "03166bf2115b3ffac50f40bbe9db200e"
server_key_iv: "e09bd9f9cf77af8cc82a5539bda442de"
server_key_version: "encryptionkey@0"
I20260619 16:58:49.492823 27967 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:49.499723 27967 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:49.501408 27967 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:49.504475 27967 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.189:44821
I20260619 16:58:49.504909 27967 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-1/data/info.pb
I20260619 16:58:49.504922 28135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.189:44821 every 8 connection(s)
I20260619 16:58:49.505663 28136 sys_catalog.cc:263] Verifying existing consensus state
I20260619 16:58:49.506042 28136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606: Bootstrap starting.
I20260619 16:58:49.508294 28136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 16:58:49.508523 28136 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606: Bootstrap complete.
I20260619 16:58:49.508713 28136 raft_consensus.cc:348] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: NON_VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: true } }
I20260619 16:58:49.508803 28136 raft_consensus.cc:729] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 674b2e8dfe9a4a8d82791cd75c0e0606, State: Initialized, Role: LEARNER
I20260619 16:58:49.508852 28136 consensus_queue.cc:260] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: NON_VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: true } }
I20260619 16:58:49.509035 28136 sys_catalog.cc:564] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:49.509006 28139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: NON_VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: true } } }
I20260619 16:58:49.509091 28139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [sys.catalog]: This master's current role is: LEARNER
I20260619 16:58:49.510474 28150 catalog_manager.cc:1259] Loaded cluster ID: 1ee2332bf6294effb4cd869600b75170
I20260619 16:58:49.510756 28150 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606: loading cluster ID for follower catalog manager: success
I20260619 16:58:49.511616 28150 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606: acquiring CA information for follower catalog manager: success
I20260619 16:58:49.511909 28150 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 16:58:49.522765 28151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.515583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50842 (local address 127.20.231.189:44821)
0619 16:58:49.515718 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:49.515721 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:49.515732 (+ 11us) server_negotiation.cc:407] Connection header received
0619 16:58:49.515759 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:49.515760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:49.515772 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:49.515867 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:49.516392 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.517445 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.518201 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.518505 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.518588 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.519981 (+ 1393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:49.519982 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:49.519983 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:49.519990 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:49.521623 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.521954 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.521955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.521955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.522022 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.522218 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.522219 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.522219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.522359 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:49.522461 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:49.522564 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 16:58:49.522648 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":62,"threads_started":1}
WARNING: no policy specified for oryx/127.20.231.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.231.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.231.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.188.keytab.
Entry for principal oryx/127.20.231.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.188.keytab.
I20260619 16:58:49.550997 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.188:46683
--webserver_interface=127.20.231.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.188.keytab
--principal=oryx/127.20.231.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:33295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:36495,127.20.231.189:44821,127.20.231.188:46683
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:49.636396 28154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:49.636706 28154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:49.636744 28154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:49.638559 28154 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:49.638623 28154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:49.638643 28154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:49.638664 28154 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:49.638690 28154 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:58:49.641081 28154 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:33295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:36495,127.20.231.189:44821,127.20.231.188:46683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.188.keytab
--principal=oryx/127.20.231.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.188:46683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.231.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:49.641383 28154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:49.641623 28154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:49.646693 28159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:49.646701 28162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:49.646711 28160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:49.647080 28154 server_base.cc:1034] running on GCE node
I20260619 16:58:49.647331 28154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:49.647643 28154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:49.648847 28154 hybrid_clock.cc:648] HybridClock initialized: now 1781888329648832 us; error 49 us; skew 500 ppm
Jun 19 16:58:49 dist-test-slave-vnbj krb5kdc[27458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888329, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:49.651391 28154 init.cc:377] Logged in from keytab as oryx/127.20.231.188@KRBTEST.COM (short username oryx)
I20260619 16:58:49.651889 28154 webserver.cc:460] Webserver started at http://127.20.231.188:36973/ using document root <none> and password file <none>
I20260619 16:58:49.652130 28154 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:49.652197 28154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:49.652345 28154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:49.653528 28154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-2/data/instance:
uuid: "24b2f725bfee4806b3880e8d28ac3b78"
format_stamp: "Formatted at 2026-06-19 16:58:49 on dist-test-slave-vnbj"
server_key: "59b32f56a614bd35935db14919cc704b"
server_key_iv: "1d22aa85669e1b97463d22f4c6d80c0e"
server_key_version: "encryptionkey@0"
I20260619 16:58:49.653893 28154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-2/wal/instance:
uuid: "24b2f725bfee4806b3880e8d28ac3b78"
format_stamp: "Formatted at 2026-06-19 16:58:49 on dist-test-slave-vnbj"
server_key: "59b32f56a614bd35935db14919cc704b"
server_key_iv: "1d22aa85669e1b97463d22f4c6d80c0e"
server_key_version: "encryptionkey@0"
I20260619 16:58:49.655308 28154 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:58:49.656071 28169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:49.656260 28154 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:49.656360 28154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-2/wal
uuid: "24b2f725bfee4806b3880e8d28ac3b78"
format_stamp: "Formatted at 2026-06-19 16:58:49 on dist-test-slave-vnbj"
server_key: "59b32f56a614bd35935db14919cc704b"
server_key_iv: "1d22aa85669e1b97463d22f4c6d80c0e"
server_key_version: "encryptionkey@0"
I20260619 16:58:49.656441 28154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:49.666488 28154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:49.668668 28154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:49.668839 28154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:49.673403 28154 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.188:46683
I20260619 16:58:49.673436 28221 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.188:46683 every 8 connection(s)
I20260619 16:58:49.673802 28154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-2/data/info.pb
I20260619 16:58:49.674795 28222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:49.676470 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 28154
I20260619 16:58:49.676396 28222 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 16:58:49 dist-test-slave-vnbj krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888329, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.188@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM
Jun 19 16:58:49 dist-test-slave-vnbj krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.231.188@KRBTEST.COM
I20260619 16:58:49.687561 27543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.677260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42634 (local address 127.20.231.190:36495)
0619 16:58:49.677305 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:49.677307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:49.677463 (+ 156us) server_negotiation.cc:407] Connection header received
0619 16:58:49.677924 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:49.677926 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:49.677937 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:49.677993 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:49.679084 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.679836 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.681091 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.681413 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.681487 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.684219 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:49.684221 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:49.684221 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:49.684230 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:49.685804 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.686378 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.686379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.686380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.686445 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.686739 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.686740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.686740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.686926 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:49.686988 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:49.687371 (+ 383us) server_negotiation.cc:299] Negotiation successful
0619 16:58:49.687474 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 16:58:49.688066 28224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.677138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36495 (local address 127.0.0.1:42634)
0619 16:58:49.677322 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:49.677363 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:49.677833 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:49.678593 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:49.678596 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:49.678837 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:49.678994 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.678999 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.679937 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.679938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.681011 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.681014 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.681511 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.681512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.681628 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.682306 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:49.682316 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:49.684127 (+ 1811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:49.685885 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.685891 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.685893 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.686274 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.686534 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.686535 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.686536 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.686662 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.686992 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:49.686998 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:49.687202 (+ 204us) client_negotiation.cc:769] Sending connection context
0619 16:58:49.687754 (+ 552us) client_negotiation.cc:240] Negotiation successful
0619 16:58:49.687863 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":107,"threads_started":1}
I20260619 16:58:49.688067 28225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.677362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50910 (local address 127.20.231.188:46683)
0619 16:58:49.677509 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:49.677514 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:49.677550 (+ 36us) server_negotiation.cc:407] Connection header received
0619 16:58:49.677849 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:49.677859 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:49.677899 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:49.678007 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:49.678595 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.680028 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.681032 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.681528 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.681646 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.684667 (+ 3021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:49.684674 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:49.684674 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:49.684681 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:49.686361 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.686736 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.686739 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.686740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.686803 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.687054 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.687055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.687056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.687209 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:49.687360 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:49.687457 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 16:58:49.687585 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":67,"threads_started":1}
I20260619 16:58:49.688627 28222 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 16:58:49 dist-test-slave-vnbj krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888329, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.188@KRBTEST.COM for oryx/127.20.231.189@KRBTEST.COM
I20260619 16:58:49.694710 28151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.688984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50856 (local address 127.20.231.189:44821)
0619 16:58:49.689027 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:49.689029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:49.689037 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:58:49.689070 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:49.689070 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:49.689081 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:49.689124 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:49.689316 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.689807 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.690473 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.690688 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.690741 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.692481 (+ 1740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:49.692482 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:49.692483 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:49.692488 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:49.693703 (+ 1215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.694045 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.694046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.694047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.694106 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.694279 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.694279 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.694280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.694375 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:49.694421 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:49.694585 (+ 164us) server_negotiation.cc:299] Negotiation successful
0619 16:58:49.694646 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 16:58:49.694723 28224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.688895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.189:44821 (local address 127.0.0.1:50856)
0619 16:58:49.688944 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:49.688955 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:49.688999 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:49.689134 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:49.689134 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:49.689237 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:49.689268 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.689269 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.689877 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.689878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.690399 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.690401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.690755 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.690756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.690804 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.691255 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:49.691260 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:49.692412 (+ 1152us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:49.693817 (+ 1405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.693818 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.693818 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.693982 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.694182 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.694183 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.694183 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.694232 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.694447 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:49.694448 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:49.694486 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 16:58:49.694567 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 16:58:49.694650 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260619 16:58:49.695221 28222 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.231.188" port: 46683 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 16:58:49 dist-test-slave-vnbj krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888329, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.188@KRBTEST.COM for oryx/127.20.231.188@KRBTEST.COM
I20260619 16:58:49.701349 28224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.695545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:46683 (local address 127.0.0.1:50914)
0619 16:58:49.695595 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:49.695606 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:49.695643 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:49.695770 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:49.695771 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:49.695876 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:49.695903 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.695904 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.696523 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.696524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.696969 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.696970 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.697330 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.697331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.697365 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.697691 (+ 326us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:49.697695 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:49.698782 (+ 1087us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:49.700394 (+ 1612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.700395 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.700396 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.700546 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.700730 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.700730 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.700731 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.700777 (+ 46us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.701094 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:49.701095 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:49.701130 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 16:58:49.701180 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 16:58:49.701254 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 16:58:49.701395 28225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.695596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50914 (local address 127.20.231.188:46683)
0619 16:58:49.695640 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:49.695642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:49.695659 (+ 17us) server_negotiation.cc:407] Connection header received
0619 16:58:49.695687 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:49.695687 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:49.695700 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:49.695766 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:49.695949 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.696445 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.697042 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.697260 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.697322 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.698911 (+ 1589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:49.698912 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:49.698913 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:49.698919 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:49.700327 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.700610 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.700611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.700611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.700667 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.700896 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.700896 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.700897 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.701024 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:49.701095 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:49.701211 (+ 116us) server_negotiation.cc:299] Negotiation successful
0619 16:58:49.701300 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":22}
I20260619 16:58:49.703013 28222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78: Bootstrap starting.
I20260619 16:58:49.703895 28222 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:49.704185 28222 log.cc:824] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:49.704948 28222 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78: No bootstrap required, opened a new log
I20260619 16:58:49.706349 28222 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 46683 } }
I20260619 16:58:49.706496 28222 raft_consensus.cc:374] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:49.706547 28222 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24b2f725bfee4806b3880e8d28ac3b78, State: Initialized, Role: FOLLOWER
I20260619 16:58:49.706700 28222 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 46683 } }
I20260619 16:58:49.707051 28222 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:49.707034 28227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 46683 } } }
I20260619 16:58:49.707142 28227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 16:58:49.708771 28238 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 16:58:49.708833 28238 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 16:58:49.711447 28154 master_runner.cc:428] Detected that this master 24b2f725bfee4806b3880e8d28ac3b78 is joining an existing cluster
I20260619 16:58:49.711493 28154 master_runner.cc:432] Initiating AddMaster RPC to add 127.20.231.188:46683
I20260619 16:58:49.719558 27543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.712855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42646 (local address 127.20.231.190:36495)
0619 16:58:49.712925 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:49.712927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:49.712935 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:58:49.712959 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:49.712960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:49.712972 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:49.713039 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:49.713341 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.714097 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.715203 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.715509 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.715576 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.716664 (+ 1088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:49.716666 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:49.716666 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:49.716673 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:49.718175 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.718599 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.718601 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.718601 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.718666 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.718922 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.718923 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.718924 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.719079 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:49.719144 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:49.719379 (+ 235us) server_negotiation.cc:299] Negotiation successful
0619 16:58:49.719464 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 16:58:49.719877 28244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.712444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36495 (local address 127.0.0.1:42646)
0619 16:58:49.712607 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:49.712619 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:49.712711 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:49.713048 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:49.713049 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:49.713200 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:49.713262 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.713263 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.714197 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.714198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.715110 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.715113 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.715585 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.715586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.715657 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.716155 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:49.716160 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:49.716568 (+ 408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:49.718271 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.718272 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.718272 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.718500 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.718744 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.718745 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.718745 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.718823 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.719173 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:49.719174 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:49.719222 (+ 48us) client_negotiation.cc:769] Sending connection context
0619 16:58:49.719331 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 16:58:49.719453 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":196,"threads_started":1}
I20260619 16:58:49.726104 28245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.712706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:46683 (local address 127.0.0.1:50922)
0619 16:58:49.718974 (+ 6268us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:49.718985 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:49.719047 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:49.719209 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:49.719210 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:49.719438 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:49.719486 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.719488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.720335 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.720335 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.721130 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.721133 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.721608 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.721609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.721669 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.722092 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:49.722097 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:49.722545 (+ 448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:49.724940 (+ 2395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.724941 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.724941 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.725150 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.725360 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.725361 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.725361 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.725425 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.725754 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:49.725755 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:49.725802 (+ 47us) client_negotiation.cc:769] Sending connection context
0619 16:58:49.725875 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 16:58:49.725976 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6245,"thread_start_us":74,"threads_started":1}
I20260619 16:58:49.726109 28225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.712725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50922 (local address 127.20.231.188:46683)
0619 16:58:49.712757 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:49.712759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:49.719055 (+ 6296us) server_negotiation.cc:407] Connection header received
0619 16:58:49.719099 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:49.719100 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:49.719110 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:49.719165 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:49.719569 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.720252 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.721219 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.721510 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.721607 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.722637 (+ 1030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:49.722644 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:49.722644 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:49.722650 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:49.724866 (+ 2216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.725229 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.725230 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.725230 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.725284 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.725492 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.725493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.725494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.725667 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:49.725731 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:49.725897 (+ 166us) server_negotiation.cc:299] Negotiation successful
0619 16:58:49.726023 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":17}
Jun 19 16:58:49 dist-test-slave-vnbj krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888328, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.190@KRBTEST.COM for oryx/127.20.231.188@KRBTEST.COM
I20260619 16:58:49.729671 28247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.722104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50924 (local address 127.20.231.188:46683)
0619 16:58:49.722265 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:49.722266 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:49.722274 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:58:49.722310 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:49.722311 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:49.722321 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:49.722397 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:49.722710 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.723473 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.724194 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.724515 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.724587 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.727214 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:49.727215 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:49.727215 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:49.727221 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:49.728598 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.728995 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.728996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.728996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.729043 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.729233 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.729233 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.729233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.729327 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:49.729393 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:49.729547 (+ 154us) server_negotiation.cc:299] Negotiation successful
0619 16:58:49.729601 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":93,"threads_started":1}
I20260619 16:58:49.729737 28060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.721960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:46683 (local address 127.0.0.1:50924)
0619 16:58:49.722009 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:49.722018 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:49.722051 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:49.722429 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:49.722431 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:49.722600 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:49.722643 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.722645 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.723596 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.723598 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.724110 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.724111 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.724602 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.724603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.724681 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.725117 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:49.725122 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:49.727098 (+ 1976us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:49.728688 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.728690 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.728690 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.728919 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.729110 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.729110 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.729111 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.729176 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.729404 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:49.729405 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:49.729451 (+ 46us) client_negotiation.cc:769] Sending connection context
0619 16:58:49.729532 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 16:58:49.729638 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 16:58:49.730218 27503 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.20.231.188:46683: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78"
member_type: NON_VOTER
last_known_addr {
host: "127.20.231.188"
port: 46683
}
attrs {
promote: true
}
}
dest_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73"
cas_config_opid_index: 5
I20260619 16:58:49.730468 27503 consensus_queue.cc:237] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: NON_VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: true } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: NON_VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: true } }
I20260619 16:58:49.730827 28061 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 24b2f725bfee4806b3880e8d28ac3b78 (127.20.231.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: NON_VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: true } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: NON_VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: true } } }
I20260619 16:58:49.732015 28061 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.20.231.188:46683
I20260619 16:58:49.732105 28061 sys_catalog.cc:455] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: NON_VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: true } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: NON_VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: true } } }
I20260619 16:58:49.732163 28061 sys_catalog.cc:458] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:49.732548 28196 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:49.733556 28196 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [term 1 FOLLOWER]: Refusing update from remote peer a91d2c9bdf114d3bb4390d2709b4cb73: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260619 16:58:49.733862 28062 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: NON_VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:49.734827 28246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.712477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.189:44821 (local address 127.0.0.1:50860)
0619 16:58:49.712820 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:49.712830 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:49.712892 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:49.713202 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:49.713203 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:49.713401 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:49.713463 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.713464 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.714418 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.714418 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.715065 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.715067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.715649 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.715650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.715721 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.731367 (+ 15646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:49.731372 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:49.731745 (+ 373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:49.733593 (+ 1848us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.733594 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.733595 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.733824 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.734164 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.734164 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.734165 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.734221 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.734486 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:49.734487 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:49.734541 (+ 54us) client_negotiation.cc:769] Sending connection context
0619 16:58:49.734601 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 16:58:49.734704 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":66,"threads_started":1}
I20260619 16:58:49.735188 28196 raft_consensus.cc:858] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 46683 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: NON_VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: NON_VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: true } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: NON_VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: true } }
W20260619 16:58:49.735247 28196 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 46683 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: NON_VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: NON_VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: true } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: NON_VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: true } }
I20260619 16:58:49.735415 28151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.712540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50860 (local address 127.20.231.189:44821)
0619 16:58:49.712588 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:49.712590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:49.713074 (+ 484us) server_negotiation.cc:407] Connection header received
0619 16:58:49.713097 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:49.713098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:49.713108 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:49.713157 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:49.713546 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.714335 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.715187 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.715530 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.715588 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.731841 (+ 16253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:49.731842 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:49.731842 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:49.731850 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:49.733503 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.734030 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.734032 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.734032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.734092 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.734283 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.734283 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.734284 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.734406 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:49.734463 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:49.735236 (+ 773us) server_negotiation.cc:299] Negotiation successful
0619 16:58:49.735321 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 16:58:49.735737 28196 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 674b2e8dfe9a4a8d82791cd75c0e0606 (127.20.231.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: NON_VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: true } } }
I20260619 16:58:49.736620 28060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.730906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.189:44821 (local address 127.0.0.1:50872)
0619 16:58:49.730958 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:49.730968 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:49.731003 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:49.731574 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:49.731575 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:49.731576 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:49.731617 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.731618 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.734010 (+ 2392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.734011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.735345 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.735348 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.736332 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.736334 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.736413 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.736439 (+ 26us) client_negotiation.cc:769] Sending connection context
0619 16:58:49.736475 (+ 36us) client_negotiation.cc:240] Negotiation successful
0619 16:58:49.736526 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 16:58:49.737112 28248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.731237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50872 (local address 127.20.231.189:44821)
0619 16:58:49.731360 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:49.731362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:49.731370 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:58:49.731393 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:49.731394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:49.731417 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:49.731680 (+ 263us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:49.731719 (+ 39us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.733919 (+ 2200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.735521 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.736227 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.736881 (+ 654us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.736957 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:49.736978 (+ 21us) server_negotiation.cc:299] Negotiation successful
0619 16:58:49.737028 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":73,"threads_started":1}
I20260619 16:58:49.737349 28154 master.cc:537] Master@127.20.231.188:46683 shutting down...
I20260619 16:58:49.737493 28227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a91d2c9bdf114d3bb4390d2709b4cb73. Latest consensus state: current_term: 1 leader_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: NON_VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: true } } }
I20260619 16:58:49.737600 28227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260619 16:58:49.737640 28227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: NON_VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: true } } }
I20260619 16:58:49.737695 28227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260619 16:58:49.737735 28227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: NON_VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: true } } }
I20260619 16:58:49.737793 28227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260619 16:58:49.737905 28110 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [term 1 LEARNER]: Refusing update from remote peer a91d2c9bdf114d3bb4390d2709b4cb73: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260619 16:58:49.738106 28062 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: NON_VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260619 16:58:49.738670 28110 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 24b2f725bfee4806b3880e8d28ac3b78 (127.20.231.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: NON_VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: true } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: NON_VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: true } } }
I20260619 16:58:49.739666 28249 mvcc.cc:205] Tried to move back new op lower bound from 7298614594540032000 to 7298614594410037248. Current Snapshot: MvccSnapshot[applied={T|T < 7298614594422362112}]
I20260619 16:58:49.740048 28139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a91d2c9bdf114d3bb4390d2709b4cb73. Latest consensus state: current_term: 1 leader_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: NON_VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: true } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: NON_VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: true } } }
I20260619 16:58:49.740132 28139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [sys.catalog]: This master's current role is: LEARNER
I20260619 16:58:49.740131 28062 sys_catalog.cc:455] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: NON_VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: true } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: NON_VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: true } } }
I20260619 16:58:49.740170 28139 sys_catalog.cc:455] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: NON_VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: true } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: NON_VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: true } } }
I20260619 16:58:49.740211 28139 sys_catalog.cc:458] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [sys.catalog]: This master's current role is: LEARNER
I20260619 16:58:49.740208 28062 sys_catalog.cc:458] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [sys.catalog]: This master's current role is: LEADER
W20260619 16:58:49.741039 27489 consensus_peers.cc:487] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 -> Peer 24b2f725bfee4806b3880e8d28ac3b78 (127.20.231.188:46683): Couldn't send request to peer 24b2f725bfee4806b3880e8d28ac3b78. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 16:58:49.741197 28061 raft_consensus.cc:1050] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73: attempting to promote NON_VOTER 674b2e8dfe9a4a8d82791cd75c0e0606 to VOTER
I20260619 16:58:49.741411 28061 consensus_queue.cc:237] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: false } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: NON_VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: true } }
I20260619 16:58:49.741907 28110 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [term 1 LEARNER]: Refusing update from remote peer a91d2c9bdf114d3bb4390d2709b4cb73: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260619 16:58:49.742158 28062 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260619 16:58:49.742240 27489 consensus_peers.cc:487] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 -> Peer 24b2f725bfee4806b3880e8d28ac3b78 (127.20.231.188:46683): Couldn't send request to peer 24b2f725bfee4806b3880e8d28ac3b78. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 16:58:49.742663 28154 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260619 16:58:49.742769 28154 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260619 16:58:49.742810 28154 tablet_replica.cc:330] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78: stopping tablet replica
I20260619 16:58:49.743010 28110 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 674b2e8dfe9a4a8d82791cd75c0e0606 (127.20.231.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: false } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: NON_VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: true } } }
W20260619 16:58:49.744084 28170 connection.cc:441] server connection from 127.0.0.1:50910 torn down before Call kudu.master.MasterService.ListTables from 127.0.0.1:50910 (request call id 25) could send its response
I20260619 16:58:49.744123 28062 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 674b2e8dfe9a4a8d82791cd75c0e0606 (127.20.231.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: false } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: NON_VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: true } } }
I20260619 16:58:49.744665 28257 sys_catalog.cc:455] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: false } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: NON_VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: true } } }
I20260619 16:58:49.745121 28257 sys_catalog.cc:458] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 16:58:49.745157 28062 sys_catalog.cc:455] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: false } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: NON_VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: true } } }
I20260619 16:58:49.745244 28062 sys_catalog.cc:458] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:49.745323 28062 sys_catalog.cc:455] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: false } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: NON_VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: true } } }
I20260619 16:58:49.745373 28062 sys_catalog.cc:458] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:49.746557 28154 master.cc:552] Master@127.20.231.188:46683 shutdown complete.
I20260619 16:58:49.746605 28154 master_runner.cc:305] Clearing existing system tablet
I20260619 16:58:49.746750 28154 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:49.746786 28154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:49.747398 28154 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:49.747881 28261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:49.748003 28154 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:49.748057 28154 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:58:49.748083 28154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-2/wal
uuid: "24b2f725bfee4806b3880e8d28ac3b78"
format_stamp: "Formatted at 2026-06-19 16:58:49 on dist-test-slave-vnbj"
server_key: "59b32f56a614bd35935db14919cc704b"
server_key_iv: "1d22aa85669e1b97463d22f4c6d80c0e"
server_key_version: "encryptionkey@0"
I20260619 16:58:49.748639 28154 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:49.750031 28154 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 16:58:49.750110 28154 log.cc:1196] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260619 16:58:49.750363 28154 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78: Deleting consensus metadata
I20260619 16:58:49.750497 28154 master_runner.cc:315] Copying system tablet from 127.20.231.190:36495
I20260619 16:58:49.751241 28154 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78: tablet copy: Beginning tablet copy session from remote peer at address 127.20.231.190:36495
I20260619 16:58:49.757718 27543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.751628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42654 (local address 127.20.231.190:36495)
0619 16:58:49.751676 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:49.751679 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:49.751738 (+ 59us) server_negotiation.cc:407] Connection header received
0619 16:58:49.751816 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:49.751817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:49.751827 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:49.751886 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:49.752220 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.752718 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.753439 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.753655 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.753728 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.754801 (+ 1073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:49.754802 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:49.754803 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:49.754810 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:49.756571 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.756931 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.756933 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.756933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.756986 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.757191 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.757192 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.757192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.757307 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:49.757380 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:49.757541 (+ 161us) server_negotiation.cc:299] Negotiation successful
0619 16:58:49.757619 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 16:58:49.757730 28267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.751554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36495 (local address 127.0.0.1:42654)
0619 16:58:49.751673 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:49.751685 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:49.751755 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:49.751891 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:49.751892 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:49.752072 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:49.752119 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.752140 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.752798 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.752799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.753354 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.753356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.753726 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.753727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.753790 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.754336 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:49.754340 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:49.754666 (+ 326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:49.756669 (+ 2003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.756670 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.756671 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.756845 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.757054 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.757055 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.757056 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.757111 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.757372 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:49.757373 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:49.757418 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 16:58:49.757516 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 16:58:49.757620 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":69,"threads_started":1}
I20260619 16:58:49.758198 27523 tablet_copy_service.cc:140] P a91d2c9bdf114d3bb4390d2709b4cb73: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 24b2f725bfee4806b3880e8d28ac3b78 ({username='oryx', principal='oryx/127.20.231.188@KRBTEST.COM'} at 127.0.0.1:42654)
I20260619 16:58:49.758268 27523 tablet_copy_service.cc:161] P a91d2c9bdf114d3bb4390d2709b4cb73: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 24b2f725bfee4806b3880e8d28ac3b78 at {username='oryx', principal='oryx/127.20.231.188@KRBTEST.COM'} at 127.0.0.1:42654: session id = 24b2f725bfee4806b3880e8d28ac3b78-00000000000000000000000000000000
I20260619 16:58:49.758884 27523 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 16:58:49.759436 28154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 16:58:49.760639 28154 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78: tablet copy: Starting download of 0 data blocks...
I20260619 16:58:49.760835 28154 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78: tablet copy: Starting download of 1 WAL segments...
I20260619 16:58:49.761719 28154 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 16:58:49.762528 27523 tablet_copy_service.cc:342] P a91d2c9bdf114d3bb4390d2709b4cb73: Request end of tablet copy session 24b2f725bfee4806b3880e8d28ac3b78-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.20.231.188@KRBTEST.COM'} at 127.0.0.1:42654
I20260619 16:58:49.762589 27523 tablet_copy_service.cc:433] P a91d2c9bdf114d3bb4390d2709b4cb73: ending tablet copy session 24b2f725bfee4806b3880e8d28ac3b78-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 24b2f725bfee4806b3880e8d28ac3b78
I20260619 16:58:49.765254 28154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:49.766649 28268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:49.766811 28154 server_base.cc:1034] running on GCE node
W20260619 16:58:49.766922 28269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:49.766960 28271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:49.767131 28154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:49.767297 28154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:49.768422 28154 hybrid_clock.cc:648] HybridClock initialized: now 1781888329768404 us; error 37 us; skew 500 ppm
Jun 19 16:58:49 dist-test-slave-vnbj krb5kdc[27458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888329, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:49.769912 28154 init.cc:377] Logged in from keytab as oryx/127.20.231.188@KRBTEST.COM (short username oryx)
I20260619 16:58:49.770172 28154 webserver.cc:460] Webserver started at http://127.20.231.188:46509/ using document root <none> and password file <none>
I20260619 16:58:49.770291 28154 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:49.770329 28154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:49.771149 28154 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:58:49.771777 28278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:49.771946 28154 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:58:49.772001 28154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-2/wal
uuid: "24b2f725bfee4806b3880e8d28ac3b78"
format_stamp: "Formatted at 2026-06-19 16:58:49 on dist-test-slave-vnbj"
server_key: "59b32f56a614bd35935db14919cc704b"
server_key_iv: "1d22aa85669e1b97463d22f4c6d80c0e"
server_key_version: "encryptionkey@0"
I20260619 16:58:49.772061 28154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:49.786945 28154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:49.788673 28154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:49.791352 28154 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.188:46683
I20260619 16:58:49.791663 28330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.188:46683 every 8 connection(s)
I20260619 16:58:49.791766 28154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/master-2/data/info.pb
I20260619 16:58:49.792450 28331 sys_catalog.cc:263] Verifying existing consensus state
I20260619 16:58:49.792865 28331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78: Bootstrap starting.
I20260619 16:58:49.795101 28331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 16:58:49.795382 28331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78: Bootstrap complete.
I20260619 16:58:49.795606 28331 raft_consensus.cc:348] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: false } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: NON_VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: true } }
I20260619 16:58:49.795710 28331 raft_consensus.cc:729] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 24b2f725bfee4806b3880e8d28ac3b78, State: Initialized, Role: LEARNER
I20260619 16:58:49.795768 28331 consensus_queue.cc:260] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: false } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: NON_VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: true } }
I20260619 16:58:49.795977 28331 sys_catalog.cc:564] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:49.795969 28335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: false } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: NON_VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: true } } }
I20260619 16:58:49.796099 28335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [sys.catalog]: This master's current role is: LEARNER
I20260619 16:58:49.797376 28346 catalog_manager.cc:1259] Loaded cluster ID: 1ee2332bf6294effb4cd869600b75170
I20260619 16:58:49.797585 28346 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78: loading cluster ID for follower catalog manager: success
I20260619 16:58:49.798920 28346 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78: acquiring CA information for follower catalog manager: success
I20260619 16:58:49.799297 28346 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 16:58:49.803047 28334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.795068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50930 (local address 127.20.231.188:46683)
0619 16:58:49.795262 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:49.795264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:49.795276 (+ 12us) server_negotiation.cc:407] Connection header received
0619 16:58:49.795299 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:49.795300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:49.795313 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:49.795394 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:49.796103 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.797127 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.798291 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.798606 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.798697 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.800265 (+ 1568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:49.800266 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:49.800266 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:49.800273 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:49.801748 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.802174 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.802175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.802176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.802266 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.802468 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.802469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.802469 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.802575 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:49.802726 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:49.802798 (+ 72us) server_negotiation.cc:299] Negotiation successful
0619 16:58:49.802921 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":82,"threads_started":1}
I20260619 16:58:49.803589 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 27565
I20260619 16:58:49.808979 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.129:36425
--local_ip_for_outbound_sockets=127.20.231.129
--tserver_master_addrs=127.20.231.190:36495
--webserver_port=38011
--webserver_interface=127.20.231.129
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.129.keytab
--principal=oryx/127.20.231.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:33295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.20.231.190:36495,127.20.231.189:44821
--tserver_master_addrs=127.20.231.190:36495,127.20.231.189:44821,127.20.231.188:46683 with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:49.894449 28347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:49.894673 28347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:49.894698 28347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:49.896368 28347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:49.896430 28347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:49.896479 28347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.129
I20260619 16:58:49.898216 28347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:33295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.129.keytab
--principal=oryx/127.20.231.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.129:36425
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.231.129
--webserver_port=38011
--tserver_master_addrs=127.20.231.190:36495,127.20.231.189:44821,127.20.231.188:46683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.129
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:49.898490 28347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:49.898734 28347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:49.904211 28353 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:49.904219 28352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:49.904340 28347 server_base.cc:1034] running on GCE node
W20260619 16:58:49.904227 28355 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:49.904692 28347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:49.904950 28347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:49.906118 28347 hybrid_clock.cc:648] HybridClock initialized: now 1781888329906115 us; error 40 us; skew 500 ppm
Jun 19 16:58:49 dist-test-slave-vnbj krb5kdc[27458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888329, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:49.908859 28347 init.cc:377] Logged in from keytab as oryx/127.20.231.129@KRBTEST.COM (short username oryx)
I20260619 16:58:49.909389 28347 webserver.cc:460] Webserver started at http://127.20.231.129:38011/ using document root <none> and password file <none>
I20260619 16:58:49.909648 28347 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:49.909731 28347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:49.911190 28347 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 16:58:49.911751 28362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:49.911926 28347 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:58:49.912025 28347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-0/wal
uuid: "46441ec526ae4a0694fe9f7907de8b8e"
format_stamp: "Formatted at 2026-06-19 16:58:48 on dist-test-slave-vnbj"
server_key: "6a06c6401df2dceb12c449f9367a8848"
server_key_iv: "a1c2968c44831fdb894a5ebd7c7ba50b"
server_key_version: "encryptionkey@0"
I20260619 16:58:49.912377 28347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:49.936563 28347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:49.938541 28347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:49.938723 28347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:49.939024 28347 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:58:49.939349 28347 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:49.939401 28347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:49.939437 28347 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:49.939464 28347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:49.945154 28347 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.129:36425
I20260619 16:58:49.945215 28475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.129:36425 every 8 connection(s)
I20260619 16:58:49.945582 28347 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
Jun 19 16:58:49 dist-test-slave-vnbj krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888329, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.188@KRBTEST.COM
Jun 19 16:58:49 dist-test-slave-vnbj krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888329, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.189@KRBTEST.COM
Jun 19 16:58:49 dist-test-slave-vnbj krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888329, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM
I20260619 16:58:49.955647 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 28347
I20260619 16:58:49.957330 28334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.946561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:48089 (local address 127.20.231.188:46683)
0619 16:58:49.946611 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:49.946613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:49.946997 (+ 384us) server_negotiation.cc:407] Connection header received
0619 16:58:49.947908 (+ 911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:49.947909 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:49.947920 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:49.947952 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:49.948800 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.949310 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.950441 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.950725 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.950794 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.953649 (+ 2855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:49.953651 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:49.953651 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:49.953660 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:49.955618 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.956217 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.956218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.956219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.956280 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.956564 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.956565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.956565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.956702 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:49.956745 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:49.957140 (+ 395us) server_negotiation.cc:299] Negotiation successful
0619 16:58:49.957237 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 16:58:49.957826 28481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.946342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:46683 (local address 127.20.231.129:48089)
0619 16:58:49.946922 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:49.946933 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:49.947776 (+ 843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:49.948042 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:49.948043 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:49.948473 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:49.948703 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.948708 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.949442 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.949443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.950345 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.950347 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.950814 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.950815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.950980 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.951579 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:49.951590 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:49.953531 (+ 1941us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:49.955804 (+ 2273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.955813 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.955815 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.956118 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.956359 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.956359 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.956360 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.956492 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.956788 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:49.956794 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:49.956977 (+ 183us) client_negotiation.cc:769] Sending connection context
0619 16:58:49.957416 (+ 439us) client_negotiation.cc:240] Negotiation successful
0619 16:58:49.957598 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"mutex_wait_us":24,"thread_start_us":94,"threads_started":1}
I20260619 16:58:49.958251 28248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.946421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:54819 (local address 127.20.231.189:44821)
0619 16:58:49.946508 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:49.946510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:49.946824 (+ 314us) server_negotiation.cc:407] Connection header received
0619 16:58:49.947945 (+ 1121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:49.947947 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:49.947959 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:49.948011 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:49.948825 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.949590 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.950719 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.951001 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.951092 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.954647 (+ 3555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:49.954648 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:49.954648 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:49.954664 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:49.956648 (+ 1984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.957075 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.957076 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.957077 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.957140 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.957349 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.957350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.957350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.957490 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:49.957532 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:49.958096 (+ 564us) server_negotiation.cc:299] Negotiation successful
0619 16:58:49.958183 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 16:58:49.958571 28482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.946342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.189:44821 (local address 127.20.231.129:54819)
0619 16:58:49.946750 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:49.946762 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:49.947758 (+ 996us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:49.948058 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:49.948058 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:49.948482 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:49.948704 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.948710 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.949691 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.949692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.950621 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.950624 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.951110 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.951111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.951179 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.952081 (+ 902us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:49.952086 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:49.954536 (+ 2450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:49.956754 (+ 2218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.956755 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.956755 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.956985 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.957218 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.957219 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.957219 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.957278 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.957585 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:49.957586 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:49.957627 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 16:58:49.958353 (+ 726us) client_negotiation.cc:240] Negotiation successful
0619 16:58:49.958458 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"mutex_wait_us":41,"thread_start_us":93,"threads_started":1}
I20260619 16:58:49.959426 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 27699
I20260619 16:58:49.964046 28478 heartbeater.cc:347] Connected to a master server at 127.20.231.189:44821
I20260619 16:58:49.964212 28478 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:49.964349 28476 heartbeater.cc:347] Connected to a master server at 127.20.231.188:46683
I20260619 16:58:49.964416 28476 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:49.964450 28478 heartbeater.cc:511] Master 127.20.231.189:44821 requested a full tablet report, sending...
I20260619 16:58:49.964490 28476 heartbeater.cc:511] Master 127.20.231.188:46683 requested a full tablet report, sending...
I20260619 16:58:49.964815 27543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.946690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:45623 (local address 127.20.231.190:36495)
0619 16:58:49.946741 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:49.946743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:49.946820 (+ 77us) server_negotiation.cc:407] Connection header received
0619 16:58:49.947873 (+ 1053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:49.947875 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:49.947888 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:49.947950 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:49.948813 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.949597 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.951151 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.951378 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.951458 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.955682 (+ 4224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:49.955684 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:49.955684 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:49.955693 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:49.957451 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.958723 (+ 1272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.958724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.958725 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.958786 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:49.964031 (+ 5245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:49.964033 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:49.964034 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:49.964212 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:49.964310 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:49.964600 (+ 290us) server_negotiation.cc:299] Negotiation successful
0619 16:58:49.964724 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 16:58:49.965016 28295 ts_manager.cc:194] Registered new tserver with Master: 46441ec526ae4a0694fe9f7907de8b8e (127.20.231.129:36425)
I20260619 16:58:49.965453 28480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:49.946342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36495 (local address 127.20.231.129:45623)
0619 16:58:49.946673 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:49.946717 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:49.947759 (+ 1042us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:49.948024 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:49.948027 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:49.948486 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:49.948704 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.948710 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.949691 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.949692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.950624 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:49.950625 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:49.951457 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:49.951458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:49.951531 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:49.952548 (+ 1017us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:49.952554 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:49.955550 (+ 2996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:49.957752 (+ 2202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.957753 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.957753 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.957986 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.958892 (+ 906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:49.958893 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:49.958894 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:49.958957 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:49.964302 (+ 5345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:49.964303 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:49.964372 (+ 69us) client_negotiation.cc:769] Sending connection context
0619 16:58:49.964953 (+ 581us) client_negotiation.cc:240] Negotiation successful
0619 16:58:49.965085 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":56,"thread_start_us":101,"threads_started":1}
I20260619 16:58:49.965471 28100 ts_manager.cc:194] Registered new tserver with Master: 46441ec526ae4a0694fe9f7907de8b8e (127.20.231.129:36425)
I20260619 16:58:49.965806 28477 heartbeater.cc:347] Connected to a master server at 127.20.231.190:36495
I20260619 16:58:49.965857 28477 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:49.965957 28477 heartbeater.cc:511] Master 127.20.231.190:36495 requested a full tablet report, sending...
I20260619 16:58:49.966282 27503 ts_manager.cc:194] Re-registered known tserver with Master: 46441ec526ae4a0694fe9f7907de8b8e (127.20.231.129:36425)
I20260619 16:58:49.966687 27503 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.231.129@KRBTEST.COM'} at 127.20.231.129:45623
I20260619 16:58:49.971549 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.130:34667
--local_ip_for_outbound_sockets=127.20.231.130
--tserver_master_addrs=127.20.231.190:36495
--webserver_port=42259
--webserver_interface=127.20.231.130
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.130.keytab
--principal=oryx/127.20.231.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:33295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.20.231.190:36495,127.20.231.189:44821
--tserver_master_addrs=127.20.231.190:36495,127.20.231.189:44821,127.20.231.188:46683 with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:50.056704 28483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:50.056951 28483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:50.056979 28483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:50.058149 28483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:50.058189 28483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:50.058216 28483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.130
I20260619 16:58:50.059798 28483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:33295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.130.keytab
--principal=oryx/127.20.231.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.130:34667
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.231.130
--webserver_port=42259
--tserver_master_addrs=127.20.231.190:36495,127.20.231.189:44821,127.20.231.188:46683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.130
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:50.059994 28483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:50.060173 28483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:50.065809 28489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:50.065965 28488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:50.065809 28491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:50.066058 28483 server_base.cc:1034] running on GCE node
I20260619 16:58:50.066306 28483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:50.066519 28483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:50.067656 28483 hybrid_clock.cc:648] HybridClock initialized: now 1781888330067638 us; error 42 us; skew 500 ppm
Jun 19 16:58:50 dist-test-slave-vnbj krb5kdc[27458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888330, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:50.070420 28483 init.cc:377] Logged in from keytab as oryx/127.20.231.130@KRBTEST.COM (short username oryx)
I20260619 16:58:50.071046 28483 webserver.cc:460] Webserver started at http://127.20.231.130:42259/ using document root <none> and password file <none>
I20260619 16:58:50.071292 28483 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:50.071376 28483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:50.072829 28483 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 16:58:50.073480 28498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:50.073750 28483 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 16:58:50.073863 28483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-1/wal
uuid: "3f0c706426784b9189a3f41b0340d499"
format_stamp: "Formatted at 2026-06-19 16:58:49 on dist-test-slave-vnbj"
server_key: "4d97b7c744a22990f69cd963f6b44e3b"
server_key_iv: "7604fe0cdf218895dc154952c2b68b66"
server_key_version: "encryptionkey@0"
I20260619 16:58:50.074246 28483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:50.098297 28483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:50.100845 28483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:50.100967 28483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:50.101212 28483 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:58:50.101532 28483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:50.101569 28483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:50.101593 28483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:50.101605 28483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:50.107566 28483 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.130:34667
I20260619 16:58:50.107645 28611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.130:34667 every 8 connection(s)
I20260619 16:58:50.108013 28483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
Jun 19 16:58:50 dist-test-slave-vnbj krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888330, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.188@KRBTEST.COM
Jun 19 16:58:50 dist-test-slave-vnbj krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888330, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM
Jun 19 16:58:50 dist-test-slave-vnbj krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888330, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.189@KRBTEST.COM
I20260619 16:58:50.117149 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 28483
I20260619 16:58:50.117311 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 27833
I20260619 16:58:50.119331 28334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:50.109269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:46913 (local address 127.20.231.188:46683)
0619 16:58:50.109323 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:50.109325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:50.109336 (+ 11us) server_negotiation.cc:407] Connection header received
0619 16:58:50.110253 (+ 917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:50.110254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:50.110268 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:50.110301 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:50.111182 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.111690 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.112739 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.113008 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.113084 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:50.115697 (+ 2613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:50.115698 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:50.115699 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:50.115705 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:50.117588 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:50.118114 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:50.118116 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:50.118117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:50.118181 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:50.118490 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:50.118491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:50.118491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:50.118649 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:50.118718 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:50.119093 (+ 375us) server_negotiation.cc:299] Negotiation successful
0619 16:58:50.119192 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 16:58:50.119868 28617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:50.108780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:46683 (local address 127.20.231.130:46913)
0619 16:58:50.108972 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:50.108996 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:50.110087 (+ 1091us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:50.110427 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:50.110432 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:50.110836 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:50.111088 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:50.111093 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.111800 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.111801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:50.112602 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:50.112604 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.113089 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.113090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:50.113220 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:50.113820 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:50.113830 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:50.115606 (+ 1776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:50.117678 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:50.117685 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:50.117687 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:50.118029 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:50.118268 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:50.118268 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:50.118269 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:50.118418 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:50.118720 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:50.118725 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:50.118931 (+ 206us) client_negotiation.cc:769] Sending connection context
0619 16:58:50.119430 (+ 499us) client_negotiation.cc:240] Negotiation successful
0619 16:58:50.119623 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":89,"threads_started":1}
I20260619 16:58:50.121685 28248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:50.109416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:48783 (local address 127.20.231.189:44821)
0619 16:58:50.109467 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:50.109468 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:50.109475 (+ 7us) server_negotiation.cc:407] Connection header received
0619 16:58:50.110380 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:50.110382 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:50.110393 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:50.110466 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:50.111202 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.111999 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.113125 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.113424 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.113467 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:50.117009 (+ 3542us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:50.117011 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:50.117011 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:50.117019 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:50.120350 (+ 3331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:50.120755 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:50.120756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:50.120756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:50.120824 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:50.121072 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:50.121073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:50.121074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:50.121229 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:50.121306 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:50.121488 (+ 182us) server_negotiation.cc:299] Negotiation successful
0619 16:58:50.121574 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 16:58:50.121850 28618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:50.109111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.189:44821 (local address 127.20.231.130:48783)
0619 16:58:50.109247 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:50.109256 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:50.110124 (+ 868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:50.110474 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:50.110474 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:50.110889 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:50.111090 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:50.111095 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.112084 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.112085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:50.113026 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:50.113028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.113545 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.113546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:50.113616 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:50.114160 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:50.114164 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:50.116912 (+ 2748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:50.120432 (+ 3520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:50.120433 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:50.120433 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:50.120655 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:50.120899 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:50.120900 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:50.120901 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:50.120972 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:50.121311 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:50.121312 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:50.121363 (+ 51us) client_negotiation.cc:769] Sending connection context
0619 16:58:50.121441 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 16:58:50.121556 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":53,"thread_start_us":69,"threads_started":1}
I20260619 16:58:50.123133 28612 heartbeater.cc:347] Connected to a master server at 127.20.231.188:46683
I20260619 16:58:50.123317 28612 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:50.123574 28612 heartbeater.cc:511] Master 127.20.231.188:46683 requested a full tablet report, sending...
I20260619 16:58:50.123618 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.131:36771
--local_ip_for_outbound_sockets=127.20.231.131
--tserver_master_addrs=127.20.231.190:36495
--webserver_port=33695
--webserver_interface=127.20.231.131
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.131.keytab
--principal=oryx/127.20.231.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:33295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.20.231.190:36495,127.20.231.189:44821
--tserver_master_addrs=127.20.231.190:36495,127.20.231.189:44821,127.20.231.188:46683 with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 16:58:50.124199 28295 ts_manager.cc:194] Registered new tserver with Master: 3f0c706426784b9189a3f41b0340d499 (127.20.231.130:34667)
I20260619 16:58:50.124497 27543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:50.108768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:59575 (local address 127.20.231.190:36495)
0619 16:58:50.108816 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:50.108818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:50.109111 (+ 293us) server_negotiation.cc:407] Connection header received
0619 16:58:50.110254 (+ 1143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:50.110255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:50.110268 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:50.110332 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:50.111203 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.111968 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.113127 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.113432 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.113468 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:50.116663 (+ 3195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:50.116665 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:50.116665 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:50.116673 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:50.121743 (+ 5070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:50.123673 (+ 1930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:50.123675 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:50.123675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:50.123747 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:50.123952 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:50.123954 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:50.123954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:50.124068 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:50.124145 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:50.124335 (+ 190us) server_negotiation.cc:299] Negotiation successful
0619 16:58:50.124409 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 16:58:50.124565 28614 heartbeater.cc:347] Connected to a master server at 127.20.231.189:44821
I20260619 16:58:50.124709 28614 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:50.124724 28616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:50.108782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36495 (local address 127.20.231.130:59575)
0619 16:58:50.108950 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:50.108995 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:50.110125 (+ 1130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:50.110429 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:50.110433 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:50.110893 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:50.111090 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:50.111095 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.112071 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.112072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:50.113026 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:50.113028 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.113545 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.113546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:50.113616 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:50.114225 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:50.114229 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:50.116565 (+ 2336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:50.123383 (+ 6818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:50.123384 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:50.123384 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:50.123570 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:50.123824 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:50.123826 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:50.123826 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:50.123879 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:50.124136 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:50.124137 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:50.124177 (+ 40us) client_negotiation.cc:769] Sending connection context
0619 16:58:50.124267 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 16:58:50.124375 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":52,"thread_start_us":81,"threads_started":1}
I20260619 16:58:50.124912 28614 heartbeater.cc:511] Master 127.20.231.189:44821 requested a full tablet report, sending...
I20260619 16:58:50.125123 28613 heartbeater.cc:347] Connected to a master server at 127.20.231.190:36495
I20260619 16:58:50.125176 28613 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:50.125258 28613 heartbeater.cc:511] Master 127.20.231.190:36495 requested a full tablet report, sending...
I20260619 16:58:50.125280 28100 ts_manager.cc:194] Registered new tserver with Master: 3f0c706426784b9189a3f41b0340d499 (127.20.231.130:34667)
I20260619 16:58:50.125481 27503 ts_manager.cc:194] Re-registered known tserver with Master: 3f0c706426784b9189a3f41b0340d499 (127.20.231.130:34667)
I20260619 16:58:50.125989 27503 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.231.130@KRBTEST.COM'} at 127.20.231.130:59575
W20260619 16:58:50.208060 28619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:50.208318 28619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:50.208366 28619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:50.209721 28619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:50.209772 28619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:50.209800 28619 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.131
I20260619 16:58:50.211463 28619 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:33295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.131.keytab
--principal=oryx/127.20.231.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.131:36771
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.231.131
--webserver_port=33695
--tserver_master_addrs=127.20.231.190:36495,127.20.231.189:44821,127.20.231.188:46683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.131
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:50.211670 28619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:50.211869 28619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:50.217551 28624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:50.217573 28625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:50.217581 28627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:50.217751 28619 server_base.cc:1034] running on GCE node
I20260619 16:58:50.218001 28619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:50.218274 28619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:50.219484 28619 hybrid_clock.cc:648] HybridClock initialized: now 1781888330219458 us; error 38 us; skew 500 ppm
Jun 19 16:58:50 dist-test-slave-vnbj krb5kdc[27458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888330, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:50.222111 28619 init.cc:377] Logged in from keytab as oryx/127.20.231.131@KRBTEST.COM (short username oryx)
I20260619 16:58:50.222640 28619 webserver.cc:460] Webserver started at http://127.20.231.131:33695/ using document root <none> and password file <none>
I20260619 16:58:50.222939 28619 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:50.223026 28619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:50.224381 28619 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:50.225003 28634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:50.225171 28619 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:58:50.225275 28619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-2/wal
uuid: "258ee1befa62410987844618763d8c16"
format_stamp: "Formatted at 2026-06-19 16:58:49 on dist-test-slave-vnbj"
server_key: "50ad76ab62f80a7163ac2735e83a0f93"
server_key_iv: "72beb9ce2bd5cc8a191b0c5b2cbef1fa"
server_key_version: "encryptionkey@0"
I20260619 16:58:50.225648 28619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:50.247105 28619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:50.249302 28619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:50.249501 28619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:50.249738 28619 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:58:50.250063 28619 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:50.250119 28619 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:50.250165 28619 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:50.250195 28619 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:50.256697 28619 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.131:36771
I20260619 16:58:50.256747 28747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.131:36771 every 8 connection(s)
I20260619 16:58:50.257091 28619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
I20260619 16:58:50.259509 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 28619
Jun 19 16:58:50 dist-test-slave-vnbj krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888330, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.189@KRBTEST.COM
Jun 19 16:58:50 dist-test-slave-vnbj krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888330, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.188@KRBTEST.COM
Jun 19 16:58:50 dist-test-slave-vnbj krb5kdc[27458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888330, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM
I20260619 16:58:50.270763 28759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:50.260687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42668 (local address 127.20.231.190:36495)
0619 16:58:50.260958 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:50.260959 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:50.261123 (+ 164us) server_negotiation.cc:407] Connection header received
0619 16:58:50.261145 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:50.261146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:50.261157 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:50.261212 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:50.262057 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.262822 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.263903 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.264186 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.264251 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:50.266236 (+ 1985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:50.266237 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:50.266238 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:50.266245 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:50.268100 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:50.269759 (+ 1659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:50.269760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:50.269761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:50.269818 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:50.270038 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:50.270040 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:50.270040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:50.270198 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:50.270312 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:50.270545 (+ 233us) server_negotiation.cc:299] Negotiation successful
0619 16:58:50.270638 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244,"thread_start_us":48,"threads_started":1}
I20260619 16:58:50.273264 27543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:50.258422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:43665 (local address 127.20.231.190:36495)
0619 16:58:50.258483 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:50.258485 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:50.258495 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:58:50.259444 (+ 949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:50.259446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:50.259459 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:50.259493 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:50.260697 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.261511 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.263123 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.263433 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.263475 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:50.268949 (+ 5474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:50.268980 (+ 31us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:50.268981 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:50.268988 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:50.271649 (+ 2661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:50.272042 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:50.272043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:50.272044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:50.272099 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:50.272463 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:50.272464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:50.272465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:50.272604 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:50.272659 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:50.273077 (+ 418us) server_negotiation.cc:299] Negotiation successful
0619 16:58:50.273150 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 16:58:50.273298 28334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:50.258216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:59883 (local address 127.20.231.188:46683)
0619 16:58:50.258381 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:50.258383 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:50.258391 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:58:50.259444 (+ 1053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:50.259446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:50.259459 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:50.259493 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:50.260703 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.261486 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.263114 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.263430 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.263474 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:50.268180 (+ 4706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:50.268181 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:50.268181 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:50.268189 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:50.270465 (+ 2276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:50.271211 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:50.271212 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:50.271212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:50.271273 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:50.272640 (+ 1367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:50.272642 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:50.272642 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:50.272791 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:50.272848 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:50.273105 (+ 257us) server_negotiation.cc:299] Negotiation successful
0619 16:58:50.273204 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260619 16:58:50.273837 28754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:50.258206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36495 (local address 127.20.231.131:43665)
0619 16:58:50.258338 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:50.258348 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:50.259323 (+ 975us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:50.259686 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:50.259691 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:50.260279 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:50.260591 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:50.260597 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.261638 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.261639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:50.263008 (+ 1369us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:50.263010 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.263559 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.263560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:50.263709 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:50.264869 (+ 1160us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:50.264874 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:50.268849 (+ 3975us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:50.271733 (+ 2884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:50.271734 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:50.271735 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:50.271955 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:50.272175 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:50.272175 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:50.272176 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:50.272375 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:50.272693 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:50.272701 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:50.272892 (+ 191us) client_negotiation.cc:769] Sending connection context
0619 16:58:50.273384 (+ 492us) client_negotiation.cc:240] Negotiation successful
0619 16:58:50.273580 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"mutex_wait_us":8,"thread_start_us":74,"threads_started":1}
I20260619 16:58:50.274284 28749 heartbeater.cc:347] Connected to a master server at 127.20.231.190:36495
I20260619 16:58:50.274327 28752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:50.257871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:46683 (local address 127.20.231.131:59883)
0619 16:58:50.258033 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:50.258071 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:50.259313 (+ 1242us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:50.259686 (+ 373us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:50.259691 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:50.260273 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:50.260590 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:50.260596 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.261628 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.261629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:50.263007 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:50.263010 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.263565 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.263566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:50.263710 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:50.265097 (+ 1387us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:50.265103 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:50.267948 (+ 2845us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:50.270555 (+ 2607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:50.270564 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:50.270567 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:50.271009 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:50.272182 (+ 1173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:50.272183 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:50.272184 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:50.272553 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:50.272880 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:50.272881 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:50.272923 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 16:58:50.273753 (+ 830us) client_negotiation.cc:240] Negotiation successful
0619 16:58:50.273863 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":111,"thread_start_us":98,"threads_started":1}
I20260619 16:58:50.274386 28749 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:50.274842 28749 heartbeater.cc:511] Master 127.20.231.190:36495 requested a full tablet report, sending...
I20260619 16:58:50.274969 28748 heartbeater.cc:347] Connected to a master server at 127.20.231.188:46683
I20260619 16:58:50.275004 28748 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:50.275058 28748 heartbeater.cc:511] Master 127.20.231.188:46683 requested a full tablet report, sending...
I20260619 16:58:50.275247 27502 ts_manager.cc:194] Re-registered known tserver with Master: 258ee1befa62410987844618763d8c16 (127.20.231.131:36771)
I20260619 16:58:50.275328 28295 ts_manager.cc:194] Registered new tserver with Master: 258ee1befa62410987844618763d8c16 (127.20.231.131:36771)
I20260619 16:58:50.273332 27503 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42668:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 16:58:50.275780 27502 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.231.131@KRBTEST.COM'} at 127.20.231.131:43665
W20260619 16:58:50.275820 27503 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:58:50.276854 28761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:50.264498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50940 (local address 127.20.231.188:46683)
0619 16:58:50.265020 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:50.265023 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:50.265192 (+ 169us) server_negotiation.cc:407] Connection header received
0619 16:58:50.265214 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:50.265215 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:50.265227 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:50.265288 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:50.266706 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.267524 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.268570 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.268895 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.268936 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:50.271638 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:50.271639 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:50.271640 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:50.271646 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:50.273586 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:50.273981 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:50.273983 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:50.273983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:50.274045 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:50.274260 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:50.274261 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:50.274261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:50.274385 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:50.274465 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:50.275931 (+ 1466us) server_negotiation.cc:299] Negotiation successful
0619 16:58:50.276021 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":70,"threads_started":1}
I20260619 16:58:50.277165 28753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:50.257870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.189:44821 (local address 127.20.231.131:57047)
0619 16:58:50.258270 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:50.258283 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:50.259331 (+ 1048us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:50.259690 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:50.259692 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:50.260273 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:50.260590 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:50.260596 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.261628 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.261629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:50.263007 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:50.263010 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.263568 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.263568 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:50.263712 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:50.264494 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:50.264506 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:50.268275 (+ 3769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:50.272812 (+ 4537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:50.272813 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:50.272814 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:50.275502 (+ 2688us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:50.276435 (+ 933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:50.276436 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:50.276436 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:50.276505 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:50.276856 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:50.276856 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:50.276895 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 16:58:50.276953 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 16:58:50.277066 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"mutex_wait_us":104,"thread_start_us":93,"threads_started":1}
I20260619 16:58:50.277458 28248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:50.257842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:57047 (local address 127.20.231.189:44821)
0619 16:58:50.257885 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:50.257888 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:50.258343 (+ 455us) server_negotiation.cc:407] Connection header received
0619 16:58:50.259461 (+ 1118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:50.259463 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:50.259476 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:50.259510 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:50.260711 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.261486 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.263115 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.263429 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.263474 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:50.268370 (+ 4896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:50.268371 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:50.268372 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:50.268379 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:50.272524 (+ 4145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:50.276292 (+ 3768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:50.276294 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:50.276295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:50.276355 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:50.276584 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:50.276585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:50.276586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:50.276745 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:50.276811 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:50.277290 (+ 479us) server_negotiation.cc:299] Negotiation successful
0619 16:58:50.277376 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 16:58:50.277905 28750 heartbeater.cc:347] Connected to a master server at 127.20.231.189:44821
I20260619 16:58:50.277956 28750 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:50.278051 28750 heartbeater.cc:511] Master 127.20.231.189:44821 requested a full tablet report, sending...
I20260619 16:58:50.278479 28100 ts_manager.cc:194] Registered new tserver with Master: 258ee1befa62410987844618763d8c16 (127.20.231.131:36771)
I20260619 16:58:50.280773 28764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:50.265069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50888 (local address 127.20.231.189:44821)
0619 16:58:50.265622 (+ 553us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:50.265624 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:50.265633 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:58:50.265654 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:50.265654 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:50.265665 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:50.265734 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:50.266325 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.267137 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.268324 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.268654 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.268727 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:50.272021 (+ 3294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:50.272022 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:50.272022 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:50.272030 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:50.278353 (+ 6323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:50.278802 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:50.278803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:50.278804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:50.278884 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:50.279138 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:50.279139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:50.279139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:50.279281 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:50.279367 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:50.280587 (+ 1220us) server_negotiation.cc:299] Negotiation successful
0619 16:58:50.280684 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":65,"threads_started":1}
I20260619 16:58:50.284560 28775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:50.280183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55860 (local address 127.20.231.130:34667)
0619 16:58:50.280622 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:50.280649 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:50.280659 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:58:50.280700 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:50.280704 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:50.280723 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:50.280828 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:50.281085 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.282266 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.283681 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.284172 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.284293 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:50.284350 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:50.284430 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 16:58:50.284470 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":89,"threads_started":1}
I20260619 16:58:50.284557 28773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:50.280124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:36771 (local address 127.0.0.1:49794)
0619 16:58:50.280450 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:50.280459 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:50.280500 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:50.281280 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:50.281281 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:50.281282 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:50.281315 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:50.281317 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.282510 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.282511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:50.283352 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:50.283355 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.284293 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.284294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:50.284357 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:50.284361 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:58:50.284392 (+ 31us) client_negotiation.cc:240] Negotiation successful
0619 16:58:50.284429 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":100,"threads_started":1}
I20260619 16:58:50.284868 28774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:50.280089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:34667 (local address 127.0.0.1:55860)
0619 16:58:50.280303 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:50.280316 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:50.280371 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:50.280825 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:50.280826 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:50.280828 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:50.280883 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:50.280886 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.282360 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.282361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:50.283598 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:50.283600 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.284286 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.284288 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:50.284350 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:50.284355 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 16:58:50.284414 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 16:58:50.284457 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":87,"thread_start_us":78,"threads_started":1}
I20260619 16:58:50.284890 28777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:50.280343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50704 (local address 127.20.231.129:36425)
0619 16:58:50.280622 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:50.280627 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:50.280970 (+ 343us) server_negotiation.cc:407] Connection header received
0619 16:58:50.281038 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:50.281041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:50.281058 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:50.281142 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:50.281353 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.282021 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.283426 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.284191 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.284281 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:50.284341 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:50.284595 (+ 254us) server_negotiation.cc:299] Negotiation successful
0619 16:58:50.284809 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":94,"threads_started":1}
I20260619 16:58:50.284971 28776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:50.280337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49794 (local address 127.20.231.131:36771)
0619 16:58:50.281107 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:50.281111 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:50.281126 (+ 15us) server_negotiation.cc:407] Connection header received
0619 16:58:50.281159 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:50.281161 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:50.281179 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:50.281258 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:50.281380 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.282416 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.283469 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.284191 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.284773 (+ 582us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:50.284832 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:50.284856 (+ 24us) server_negotiation.cc:299] Negotiation successful
0619 16:58:50.284895 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":739,"thread_start_us":81,"threads_started":1}
I20260619 16:58:50.285259 28772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:50.280132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:36425 (local address 127.0.0.1:50704)
0619 16:58:50.280910 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:50.280920 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:50.280969 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:50.281221 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:50.281222 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:50.281224 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:50.281273 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:50.281275 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.282114 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.282115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:50.283338 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:50.283341 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.284371 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.284372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:50.284424 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:50.284427 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:58:50.284471 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 16:58:50.284507 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":758,"mutex_wait_us":1,"thread_start_us":95,"threads_started":1}
I20260619 16:58:50.285632 28682 tablet_service.cc:1467] Processing CreateTablet for tablet b231f76ed84e437095807a5bef83f470 (DEFAULT_TABLE table=test-table [id=5dc7dd4bbc894bc4973343a4ff447a43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:50.285714 28546 tablet_service.cc:1467] Processing CreateTablet for tablet b231f76ed84e437095807a5bef83f470 (DEFAULT_TABLE table=test-table [id=5dc7dd4bbc894bc4973343a4ff447a43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:50.285929 28682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b231f76ed84e437095807a5bef83f470. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:50.286072 28546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b231f76ed84e437095807a5bef83f470. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:50.287770 28410 tablet_service.cc:1467] Processing CreateTablet for tablet b231f76ed84e437095807a5bef83f470 (DEFAULT_TABLE table=test-table [id=5dc7dd4bbc894bc4973343a4ff447a43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:50.288118 28410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b231f76ed84e437095807a5bef83f470. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:50.289076 28778 tablet_bootstrap.cc:492] T b231f76ed84e437095807a5bef83f470 P 3f0c706426784b9189a3f41b0340d499: Bootstrap starting.
I20260619 16:58:50.289428 28779 tablet_bootstrap.cc:492] T b231f76ed84e437095807a5bef83f470 P 258ee1befa62410987844618763d8c16: Bootstrap starting.
I20260619 16:58:50.289978 28778 tablet_bootstrap.cc:654] T b231f76ed84e437095807a5bef83f470 P 3f0c706426784b9189a3f41b0340d499: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:50.289999 28779 tablet_bootstrap.cc:654] T b231f76ed84e437095807a5bef83f470 P 258ee1befa62410987844618763d8c16: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:50.290333 28778 log.cc:824] T b231f76ed84e437095807a5bef83f470 P 3f0c706426784b9189a3f41b0340d499: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:50.290381 28779 log.cc:824] T b231f76ed84e437095807a5bef83f470 P 258ee1befa62410987844618763d8c16: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:50.291164 28778 tablet_bootstrap.cc:492] T b231f76ed84e437095807a5bef83f470 P 3f0c706426784b9189a3f41b0340d499: No bootstrap required, opened a new log
I20260619 16:58:50.291186 28779 tablet_bootstrap.cc:492] T b231f76ed84e437095807a5bef83f470 P 258ee1befa62410987844618763d8c16: No bootstrap required, opened a new log
I20260619 16:58:50.291265 28779 ts_tablet_manager.cc:1397] T b231f76ed84e437095807a5bef83f470 P 258ee1befa62410987844618763d8c16: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:58:50.291266 28778 ts_tablet_manager.cc:1397] T b231f76ed84e437095807a5bef83f470 P 3f0c706426784b9189a3f41b0340d499: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:58:50.291679 28782 tablet_bootstrap.cc:492] T b231f76ed84e437095807a5bef83f470 P 46441ec526ae4a0694fe9f7907de8b8e: Bootstrap starting.
I20260619 16:58:50.292363 28782 tablet_bootstrap.cc:654] T b231f76ed84e437095807a5bef83f470 P 46441ec526ae4a0694fe9f7907de8b8e: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:50.292683 28782 log.cc:824] T b231f76ed84e437095807a5bef83f470 P 46441ec526ae4a0694fe9f7907de8b8e: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:50.293694 28782 tablet_bootstrap.cc:492] T b231f76ed84e437095807a5bef83f470 P 46441ec526ae4a0694fe9f7907de8b8e: No bootstrap required, opened a new log
I20260619 16:58:50.293771 28782 ts_tablet_manager.cc:1397] T b231f76ed84e437095807a5bef83f470 P 46441ec526ae4a0694fe9f7907de8b8e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:58:50.293715 28779 raft_consensus.cc:348] T b231f76ed84e437095807a5bef83f470 P 258ee1befa62410987844618763d8c16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c706426784b9189a3f41b0340d499" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34667 } } peers { permanent_uuid: "258ee1befa62410987844618763d8c16" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36771 } } peers { permanent_uuid: "46441ec526ae4a0694fe9f7907de8b8e" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 36425 } }
I20260619 16:58:50.293938 28779 raft_consensus.cc:374] T b231f76ed84e437095807a5bef83f470 P 258ee1befa62410987844618763d8c16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:50.293982 28779 raft_consensus.cc:729] T b231f76ed84e437095807a5bef83f470 P 258ee1befa62410987844618763d8c16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 258ee1befa62410987844618763d8c16, State: Initialized, Role: FOLLOWER
I20260619 16:58:50.293955 28778 raft_consensus.cc:348] T b231f76ed84e437095807a5bef83f470 P 3f0c706426784b9189a3f41b0340d499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c706426784b9189a3f41b0340d499" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34667 } } peers { permanent_uuid: "258ee1befa62410987844618763d8c16" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36771 } } peers { permanent_uuid: "46441ec526ae4a0694fe9f7907de8b8e" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 36425 } }
I20260619 16:58:50.294117 28778 raft_consensus.cc:374] T b231f76ed84e437095807a5bef83f470 P 3f0c706426784b9189a3f41b0340d499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:50.294353 28779 consensus_queue.cc:260] T b231f76ed84e437095807a5bef83f470 P 258ee1befa62410987844618763d8c16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c706426784b9189a3f41b0340d499" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34667 } } peers { permanent_uuid: "258ee1befa62410987844618763d8c16" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36771 } } peers { permanent_uuid: "46441ec526ae4a0694fe9f7907de8b8e" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 36425 } }
I20260619 16:58:50.294601 28778 raft_consensus.cc:729] T b231f76ed84e437095807a5bef83f470 P 3f0c706426784b9189a3f41b0340d499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f0c706426784b9189a3f41b0340d499, State: Initialized, Role: FOLLOWER
I20260619 16:58:50.294668 28779 ts_tablet_manager.cc:1428] T b231f76ed84e437095807a5bef83f470 P 258ee1befa62410987844618763d8c16: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 16:58:50.294808 28749 heartbeater.cc:503] Master 127.20.231.190:36495 was elected leader, sending a full tablet report...
I20260619 16:58:50.294790 28778 consensus_queue.cc:260] T b231f76ed84e437095807a5bef83f470 P 3f0c706426784b9189a3f41b0340d499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c706426784b9189a3f41b0340d499" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34667 } } peers { permanent_uuid: "258ee1befa62410987844618763d8c16" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36771 } } peers { permanent_uuid: "46441ec526ae4a0694fe9f7907de8b8e" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 36425 } }
I20260619 16:58:50.295069 28778 ts_tablet_manager.cc:1428] T b231f76ed84e437095807a5bef83f470 P 3f0c706426784b9189a3f41b0340d499: Time spent starting tablet: real 0.004s user 0.001s sys 0.002s
I20260619 16:58:50.295713 28613 heartbeater.cc:503] Master 127.20.231.190:36495 was elected leader, sending a full tablet report...
I20260619 16:58:50.297303 28782 raft_consensus.cc:348] T b231f76ed84e437095807a5bef83f470 P 46441ec526ae4a0694fe9f7907de8b8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c706426784b9189a3f41b0340d499" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34667 } } peers { permanent_uuid: "258ee1befa62410987844618763d8c16" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36771 } } peers { permanent_uuid: "46441ec526ae4a0694fe9f7907de8b8e" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 36425 } }
I20260619 16:58:50.297477 28782 raft_consensus.cc:374] T b231f76ed84e437095807a5bef83f470 P 46441ec526ae4a0694fe9f7907de8b8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:50.297541 28782 raft_consensus.cc:729] T b231f76ed84e437095807a5bef83f470 P 46441ec526ae4a0694fe9f7907de8b8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46441ec526ae4a0694fe9f7907de8b8e, State: Initialized, Role: FOLLOWER
I20260619 16:58:50.297657 28782 consensus_queue.cc:260] T b231f76ed84e437095807a5bef83f470 P 46441ec526ae4a0694fe9f7907de8b8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c706426784b9189a3f41b0340d499" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34667 } } peers { permanent_uuid: "258ee1befa62410987844618763d8c16" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36771 } } peers { permanent_uuid: "46441ec526ae4a0694fe9f7907de8b8e" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 36425 } }
I20260619 16:58:50.297935 28782 ts_tablet_manager.cc:1428] T b231f76ed84e437095807a5bef83f470 P 46441ec526ae4a0694fe9f7907de8b8e: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260619 16:58:50.298013 28477 heartbeater.cc:503] Master 127.20.231.190:36495 was elected leader, sending a full tablet report...
I20260619 16:58:50.312507 28784 raft_consensus.cc:482] T b231f76ed84e437095807a5bef83f470 P 258ee1befa62410987844618763d8c16 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:58:50.312635 28784 raft_consensus.cc:504] T b231f76ed84e437095807a5bef83f470 P 258ee1befa62410987844618763d8c16 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c706426784b9189a3f41b0340d499" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34667 } } peers { permanent_uuid: "258ee1befa62410987844618763d8c16" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36771 } } peers { permanent_uuid: "46441ec526ae4a0694fe9f7907de8b8e" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 36425 } }
I20260619 16:58:50.313016 28784 leader_election.cc:288] T b231f76ed84e437095807a5bef83f470 P 258ee1befa62410987844618763d8c16 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f0c706426784b9189a3f41b0340d499 (127.20.231.130:34667), 46441ec526ae4a0694fe9f7907de8b8e (127.20.231.129:36425)
I20260619 16:58:50.316905 28752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:50.313278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:34667 (local address 127.20.231.131:60091)
0619 16:58:50.313335 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:50.313345 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:50.313380 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:50.313543 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:50.313544 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:50.313546 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:50.313625 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:50.313628 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.314638 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.314639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:50.315832 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:50.315835 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.316549 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.316550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:50.316681 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:50.316686 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 16:58:50.316744 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 16:58:50.316788 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260619 16:58:50.316905 28775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:50.313332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:60091 (local address 127.20.231.130:34667)
0619 16:58:50.313391 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:50.313393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:50.313413 (+ 20us) server_negotiation.cc:407] Connection header received
0619 16:58:50.313448 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:50.313450 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:50.313460 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:50.313524 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:50.313714 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.314553 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.315919 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.316462 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.316534 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:50.316542 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:50.316772 (+ 230us) server_negotiation.cc:299] Negotiation successful
0619 16:58:50.316822 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 16:58:50.317003 28777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:50.313409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:43233 (local address 127.20.231.129:36425)
0619 16:58:50.313465 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:50.313467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:50.313480 (+ 13us) server_negotiation.cc:407] Connection header received
0619 16:58:50.313520 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:50.313521 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:50.313531 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:50.313583 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:50.313891 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.314614 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.316014 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.316562 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.316656 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:50.316665 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:50.316871 (+ 206us) server_negotiation.cc:299] Negotiation successful
0619 16:58:50.316917 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 16:58:50.317250 28753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:50.313177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:36425 (local address 127.20.231.131:43233)
0619 16:58:50.313290 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:50.313304 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:50.313368 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:50.313612 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:50.313613 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:50.313614 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:50.313742 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:50.313744 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.314738 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.314739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:50.315911 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:50.315914 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.316668 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.316669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:50.316779 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:50.316783 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:58:50.316845 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 16:58:50.316897 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91}
I20260619 16:58:50.317370 28566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b231f76ed84e437095807a5bef83f470" candidate_uuid: "258ee1befa62410987844618763d8c16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f0c706426784b9189a3f41b0340d499" is_pre_election: true
I20260619 16:58:50.317562 28566 raft_consensus.cc:2454] T b231f76ed84e437095807a5bef83f470 P 3f0c706426784b9189a3f41b0340d499 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 258ee1befa62410987844618763d8c16 in term 0.
I20260619 16:58:50.317636 28430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b231f76ed84e437095807a5bef83f470" candidate_uuid: "258ee1befa62410987844618763d8c16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46441ec526ae4a0694fe9f7907de8b8e" is_pre_election: true
I20260619 16:58:50.317783 28430 raft_consensus.cc:2454] T b231f76ed84e437095807a5bef83f470 P 46441ec526ae4a0694fe9f7907de8b8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 258ee1befa62410987844618763d8c16 in term 0.
I20260619 16:58:50.318017 28636 leader_election.cc:302] T b231f76ed84e437095807a5bef83f470 P 258ee1befa62410987844618763d8c16 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 258ee1befa62410987844618763d8c16, 3f0c706426784b9189a3f41b0340d499; no voters:
I20260619 16:58:50.318210 28784 raft_consensus.cc:2781] T b231f76ed84e437095807a5bef83f470 P 258ee1befa62410987844618763d8c16 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:58:50.318295 28784 raft_consensus.cc:482] T b231f76ed84e437095807a5bef83f470 P 258ee1befa62410987844618763d8c16 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:58:50.318315 28784 raft_consensus.cc:3037] T b231f76ed84e437095807a5bef83f470 P 258ee1befa62410987844618763d8c16 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:50.319280 28784 raft_consensus.cc:504] T b231f76ed84e437095807a5bef83f470 P 258ee1befa62410987844618763d8c16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c706426784b9189a3f41b0340d499" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34667 } } peers { permanent_uuid: "258ee1befa62410987844618763d8c16" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36771 } } peers { permanent_uuid: "46441ec526ae4a0694fe9f7907de8b8e" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 36425 } }
I20260619 16:58:50.319450 28784 leader_election.cc:288] T b231f76ed84e437095807a5bef83f470 P 258ee1befa62410987844618763d8c16 [CANDIDATE]: Term 1 election: Requested vote from peers 3f0c706426784b9189a3f41b0340d499 (127.20.231.130:34667), 46441ec526ae4a0694fe9f7907de8b8e (127.20.231.129:36425)
I20260619 16:58:50.319605 28566 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b231f76ed84e437095807a5bef83f470" candidate_uuid: "258ee1befa62410987844618763d8c16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f0c706426784b9189a3f41b0340d499"
I20260619 16:58:50.319641 28430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b231f76ed84e437095807a5bef83f470" candidate_uuid: "258ee1befa62410987844618763d8c16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46441ec526ae4a0694fe9f7907de8b8e"
I20260619 16:58:50.319696 28566 raft_consensus.cc:3037] T b231f76ed84e437095807a5bef83f470 P 3f0c706426784b9189a3f41b0340d499 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:50.319732 28430 raft_consensus.cc:3037] T b231f76ed84e437095807a5bef83f470 P 46441ec526ae4a0694fe9f7907de8b8e [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:50.320675 28566 raft_consensus.cc:2454] T b231f76ed84e437095807a5bef83f470 P 3f0c706426784b9189a3f41b0340d499 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 258ee1befa62410987844618763d8c16 in term 1.
I20260619 16:58:50.320675 28430 raft_consensus.cc:2454] T b231f76ed84e437095807a5bef83f470 P 46441ec526ae4a0694fe9f7907de8b8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 258ee1befa62410987844618763d8c16 in term 1.
I20260619 16:58:50.320863 28636 leader_election.cc:302] T b231f76ed84e437095807a5bef83f470 P 258ee1befa62410987844618763d8c16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 258ee1befa62410987844618763d8c16, 3f0c706426784b9189a3f41b0340d499; no voters:
I20260619 16:58:50.320991 28784 raft_consensus.cc:2781] T b231f76ed84e437095807a5bef83f470 P 258ee1befa62410987844618763d8c16 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:50.321146 28784 raft_consensus.cc:686] T b231f76ed84e437095807a5bef83f470 P 258ee1befa62410987844618763d8c16 [term 1 LEADER]: Becoming Leader. State: Replica: 258ee1befa62410987844618763d8c16, State: Running, Role: LEADER
I20260619 16:58:50.321250 28784 consensus_queue.cc:237] T b231f76ed84e437095807a5bef83f470 P 258ee1befa62410987844618763d8c16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c706426784b9189a3f41b0340d499" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34667 } } peers { permanent_uuid: "258ee1befa62410987844618763d8c16" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36771 } } peers { permanent_uuid: "46441ec526ae4a0694fe9f7907de8b8e" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 36425 } }
I20260619 16:58:50.322190 27503 catalog_manager.cc:5495] T b231f76ed84e437095807a5bef83f470 P 258ee1befa62410987844618763d8c16 reported cstate change: term changed from 0 to 1, leader changed from <none> to 258ee1befa62410987844618763d8c16 (127.20.231.131). New cstate: current_term: 1 leader_uuid: "258ee1befa62410987844618763d8c16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c706426784b9189a3f41b0340d499" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "258ee1befa62410987844618763d8c16" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36771 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46441ec526ae4a0694fe9f7907de8b8e" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 36425 } health_report { overall_health: UNKNOWN } } }
I20260619 16:58:50.332193 28761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:50.328999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50946 (local address 127.20.231.188:46683)
0619 16:58:50.329048 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:50.329051 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:50.329060 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:58:50.329136 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:50.329137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:50.329157 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:50.329264 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:50.329432 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.330150 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.331363 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.331812 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.331915 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:50.331997 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:50.332047 (+ 50us) server_negotiation.cc:299] Negotiation successful
0619 16:58:50.332108 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 16:58:50.332194 28772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:50.328917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:46683 (local address 127.0.0.1:50946)
0619 16:58:50.328988 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:50.329002 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:50.329058 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:50.329283 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:50.329284 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:50.329286 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:50.329354 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:50.329356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.330248 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.330249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:50.331263 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:50.331266 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.331911 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.331912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:50.331969 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:50.331978 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 16:58:50.332038 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 16:58:50.332128 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260619 16:58:50.332552 28305 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.10] Dedup: 1.7->[1.8-1.10]
I20260619 16:58:50.332984 28790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a91d2c9bdf114d3bb4390d2709b4cb73. Latest consensus state: current_term: 1 leader_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: false } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: NON_VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: true } } }
I20260619 16:58:50.333109 28790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [sys.catalog]: This master's current role is: LEARNER
I20260619 16:58:50.333863 28776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:50.331075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49798 (local address 127.20.231.131:36771)
0619 16:58:50.331156 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:50.331159 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:50.331168 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:58:50.331204 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:50.331205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:50.331224 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:50.331316 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:58:50.331457 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.331955 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.332927 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.333208 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.333277 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:50.333495 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:58:50.333643 (+ 148us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:58:50.333721 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:50.333765 (+ 44us) server_negotiation.cc:299] Negotiation successful
0619 16:58:50.333787 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 16:58:50.335899 28566 raft_consensus.cc:1261] T b231f76ed84e437095807a5bef83f470 P 3f0c706426784b9189a3f41b0340d499 [term 1 FOLLOWER]: Refusing update from remote peer 258ee1befa62410987844618763d8c16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:50.335940 28430 raft_consensus.cc:1261] T b231f76ed84e437095807a5bef83f470 P 46441ec526ae4a0694fe9f7907de8b8e [term 1 FOLLOWER]: Refusing update from remote peer 258ee1befa62410987844618763d8c16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:50.336321 28787 consensus_queue.cc:1035] T b231f76ed84e437095807a5bef83f470 P 258ee1befa62410987844618763d8c16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f0c706426784b9189a3f41b0340d499" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:50.336499 28784 consensus_queue.cc:1035] T b231f76ed84e437095807a5bef83f470 P 258ee1befa62410987844618763d8c16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46441ec526ae4a0694fe9f7907de8b8e" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 36425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:50.339262 28796 mvcc.cc:205] Tried to move back new op lower bound from 7298614601052540928 to 7298614600996687872. Current Snapshot: MvccSnapshot[applied={T|T < 7298614601052540928}]
I20260619 16:58:50.340016 28795 mvcc.cc:205] Tried to move back new op lower bound from 7298614601052540928 to 7298614600996687872. Current Snapshot: MvccSnapshot[applied={T|T < 7298614601052540928}]
I20260619 16:58:50.342518 27503 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:58:50.342615 27503 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:58:50.343210 28771 sys_catalog.cc:455] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: false } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: NON_VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: true } } }
I20260619 16:58:50.343349 28771 sys_catalog.cc:458] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:50.344282 27503 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73: Sending DeleteTablet for 3 replicas of tablet b231f76ed84e437095807a5bef83f470
I20260619 16:58:50.344590 28546 tablet_service.cc:1514] Processing DeleteTablet for tablet b231f76ed84e437095807a5bef83f470 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:50 UTC) from {username='oryx', principal='oryx/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:55860
I20260619 16:58:50.344645 28804 raft_consensus.cc:1050] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73: attempting to promote NON_VOTER 24b2f725bfee4806b3880e8d28ac3b78 to VOTER
I20260619 16:58:50.344630 28410 tablet_service.cc:1514] Processing DeleteTablet for tablet b231f76ed84e437095807a5bef83f470 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:50 UTC) from {username='oryx', principal='oryx/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:50704
I20260619 16:58:50.344770 28682 tablet_service.cc:1514] Processing DeleteTablet for tablet b231f76ed84e437095807a5bef83f470 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:50 UTC) from {username='oryx', principal='oryx/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:49794
I20260619 16:58:50.344806 28804 consensus_queue.cc:237] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.11, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: false } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: false } }
I20260619 16:58:50.345407 28305 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [term 1 LEARNER]: Refusing update from remote peer a91d2c9bdf114d3bb4390d2709b4cb73: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20260619 16:58:50.345726 28110 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [term 1 FOLLOWER]: Refusing update from remote peer a91d2c9bdf114d3bb4390d2709b4cb73: Log matching property violated. Preceding OpId in replica: term: 1 index: 11. Preceding OpId from leader: term: 1 index: 12. (index mismatch)
I20260619 16:58:50.345733 28771 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260619 16:58:50.345935 28771 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a91d2c9bdf114d3bb4390d2709b4cb73 [LEADER]: Connected to new peer: Peer: permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 12, Last known committed idx: 11, Time since last communication: 0.000s
I20260619 16:58:50.346191 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 27471
I20260619 16:58:50.346994 28807 tablet_replica.cc:330] T b231f76ed84e437095807a5bef83f470 P 3f0c706426784b9189a3f41b0340d499: stopping tablet replica
I20260619 16:58:50.347114 28807 raft_consensus.cc:2229] T b231f76ed84e437095807a5bef83f470 P 3f0c706426784b9189a3f41b0340d499 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:58:50.347283 28806 tablet_replica.cc:330] T b231f76ed84e437095807a5bef83f470 P 46441ec526ae4a0694fe9f7907de8b8e: stopping tablet replica
I20260619 16:58:50.347425 28806 raft_consensus.cc:2229] T b231f76ed84e437095807a5bef83f470 P 46441ec526ae4a0694fe9f7907de8b8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:58:50.347658 28807 raft_consensus.cc:2258] T b231f76ed84e437095807a5bef83f470 P 3f0c706426784b9189a3f41b0340d499 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:50.347864 28806 raft_consensus.cc:2258] T b231f76ed84e437095807a5bef83f470 P 46441ec526ae4a0694fe9f7907de8b8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:50.348433 28807 ts_tablet_manager.cc:1905] T b231f76ed84e437095807a5bef83f470 P 3f0c706426784b9189a3f41b0340d499: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:50.350597 28807 ts_tablet_manager.cc:1918] T b231f76ed84e437095807a5bef83f470 P 3f0c706426784b9189a3f41b0340d499: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:58:50.350680 28807 log.cc:1196] T b231f76ed84e437095807a5bef83f470 P 3f0c706426784b9189a3f41b0340d499: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-1/wal/wals/b231f76ed84e437095807a5bef83f470
I20260619 16:58:50.351058 28807 ts_tablet_manager.cc:1939] T b231f76ed84e437095807a5bef83f470 P 3f0c706426784b9189a3f41b0340d499: Deleting consensus metadata
I20260619 16:58:50.351347 28806 ts_tablet_manager.cc:1905] T b231f76ed84e437095807a5bef83f470 P 46441ec526ae4a0694fe9f7907de8b8e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 16:58:50.352263 28282 connection.cc:535] server connection from 127.0.0.1:50946 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260619 16:58:50.352432 28086 connection.cc:535] server connection from 127.0.0.1:50872 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260619 16:58:50.352771 28501 connection.cc:535] server connection from 127.0.0.1:55860 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 16:58:50.353467 28806 ts_tablet_manager.cc:1918] T b231f76ed84e437095807a5bef83f470 P 46441ec526ae4a0694fe9f7907de8b8e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:58:50.353557 28806 log.cc:1196] T b231f76ed84e437095807a5bef83f470 P 46441ec526ae4a0694fe9f7907de8b8e: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-0/wal/wals/b231f76ed84e437095807a5bef83f470
I20260619 16:58:50.353790 28806 ts_tablet_manager.cc:1939] T b231f76ed84e437095807a5bef83f470 P 46441ec526ae4a0694fe9f7907de8b8e: Deleting consensus metadata
W20260619 16:58:50.354179 28363 connection.cc:441] server connection from 127.0.0.1:50704 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50704 (request call id 1) could send its response
I20260619 16:58:50.354233 28808 tablet_replica.cc:330] T b231f76ed84e437095807a5bef83f470 P 258ee1befa62410987844618763d8c16: stopping tablet replica
I20260619 16:58:50.354348 28808 raft_consensus.cc:2229] T b231f76ed84e437095807a5bef83f470 P 258ee1befa62410987844618763d8c16 [term 1 LEADER]: Raft consensus shutting down.
I20260619 16:58:50.354488 28808 raft_consensus.cc:2258] T b231f76ed84e437095807a5bef83f470 P 258ee1befa62410987844618763d8c16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:50.354982 28808 ts_tablet_manager.cc:1905] T b231f76ed84e437095807a5bef83f470 P 258ee1befa62410987844618763d8c16: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:50.356647 28808 ts_tablet_manager.cc:1918] T b231f76ed84e437095807a5bef83f470 P 258ee1befa62410987844618763d8c16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:58:50.356758 28808 log.cc:1196] T b231f76ed84e437095807a5bef83f470 P 258ee1befa62410987844618763d8c16: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-2/wal/wals/b231f76ed84e437095807a5bef83f470
I20260619 16:58:50.357095 28808 ts_tablet_manager.cc:1939] T b231f76ed84e437095807a5bef83f470 P 258ee1befa62410987844618763d8c16: Deleting consensus metadata
W20260619 16:58:50.357496 28638 connection.cc:441] server connection from 127.0.0.1:49794 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:49794 (request call id 1) could send its response
I20260619 16:58:50.362787 28761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:50.354672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50952 (local address 127.20.231.188:46683)
0619 16:58:50.354715 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:50.354718 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:50.354945 (+ 227us) server_negotiation.cc:407] Connection header received
0619 16:58:50.355011 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:50.355013 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:50.355024 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:50.355091 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:50.355788 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.356585 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.357783 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.358095 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.358135 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:50.359844 (+ 1709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:50.359845 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:50.359845 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:50.359854 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:50.361553 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:50.361916 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:50.361917 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:50.361918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:50.361975 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:50.362214 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:50.362215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:50.362215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:50.362361 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:50.362470 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:50.362582 (+ 112us) server_negotiation.cc:299] Negotiation successful
0619 16:58:50.362676 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 16:58:50.363931 28764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:50.356687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50894 (local address 127.20.231.189:44821)
0619 16:58:50.356731 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:50.356733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:50.356983 (+ 250us) server_negotiation.cc:407] Connection header received
0619 16:58:50.357085 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:50.357086 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:50.357102 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:50.357133 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:50.357887 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.358369 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.359179 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:50.359499 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:50.359567 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:50.360935 (+ 1368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:50.360936 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:50.360936 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:50.360944 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:50.362632 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:50.363044 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:50.363045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:50.363045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:50.363105 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:50.363319 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:50.363319 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:50.363320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:50.363462 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:50.363518 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:50.363702 (+ 184us) server_negotiation.cc:299] Negotiation successful
0619 16:58:50.363783 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 16:58:51.340905 28815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.340483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36495 (local address 127.20.231.131:50361)
0619 16:58:51.340734 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:51.340834 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:36495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":78,"threads_started":1}
I20260619 16:58:51.340937 28816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.340544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36495 (local address 127.20.231.130:49723)
0619 16:58:51.340798 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:51.340888 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:36495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":140,"threads_started":1}
W20260619 16:58:51.341221 28749 heartbeater.cc:644] Failed to heartbeat to 127.20.231.190:36495 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.231.190:36495: connect: Connection refused (error 111)
W20260619 16:58:51.341441 28613 heartbeater.cc:644] Failed to heartbeat to 127.20.231.190:36495 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.231.190:36495: connect: Connection refused (error 111)
I20260619 16:58:51.341777 28817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.341530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36495 (local address 127.20.231.129:49835)
0619 16:58:51.341658 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:51.341720 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:36495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":59,"threads_started":1}
W20260619 16:58:51.342028 28477 heartbeater.cc:644] Failed to heartbeat to 127.20.231.190:36495 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.20.231.190:36495: connect: Connection refused (error 111)
I20260619 16:58:51.781985 28818 raft_consensus.cc:482] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a91d2c9bdf114d3bb4390d2709b4cb73)
I20260619 16:58:51.782123 28818 raft_consensus.cc:504] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: false } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: false } }
I20260619 16:58:51.782394 28818 leader_election.cc:288] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a91d2c9bdf114d3bb4390d2709b4cb73 (127.20.231.190:36495), 674b2e8dfe9a4a8d82791cd75c0e0606 (127.20.231.189:44821)
I20260619 16:58:51.782830 28819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.782571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36495 (local address 127.0.0.1:43136)
0619 16:58:51.782713 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:51.782774 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:36495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":73,"threads_started":1}
W20260619 16:58:51.783031 28279 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.231.190:36495: connect: Connection refused (error 111)
I20260619 16:58:51.783533 28819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.783389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36495 (local address 127.0.0.1:43148)
0619 16:58:51.783459 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:51.783507 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:36495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260619 16:58:51.783722 28279 leader_election.cc:334] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a91d2c9bdf114d3bb4390d2709b4cb73 (127.20.231.190:36495): Network error: Client connection negotiation failed: client connection to 127.20.231.190:36495: connect: Connection refused (error 111)
I20260619 16:58:51.785756 28821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.782675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50902 (local address 127.20.231.189:44821)
0619 16:58:51.782823 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:51.782826 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:51.782840 (+ 14us) server_negotiation.cc:407] Connection header received
0619 16:58:51.782909 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:51.782911 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:51.782925 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:51.783007 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:51.783179 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:51.783961 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:51.785071 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:51.785443 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:51.785506 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:51.785513 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:51.785667 (+ 154us) server_negotiation.cc:299] Negotiation successful
0619 16:58:51.785701 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":70,"threads_started":1}
I20260619 16:58:51.785794 28820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.782585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.189:44821 (local address 127.0.0.1:50902)
0619 16:58:51.782721 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:51.782734 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:51.782816 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:51.783010 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:51.783011 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:51.783013 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:51.783081 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:51.783085 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:51.784041 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:51.784043 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:51.784986 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:51.784989 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:51.785537 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:51.785539 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:51.785594 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:51.785599 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 16:58:51.785656 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 16:58:51.785693 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":58,"threads_started":1}
I20260619 16:58:51.786049 28110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "24b2f725bfee4806b3880e8d28ac3b78" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" is_pre_election: true
I20260619 16:58:51.786340 28280 leader_election.cc:302] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 24b2f725bfee4806b3880e8d28ac3b78; no voters: 674b2e8dfe9a4a8d82791cd75c0e0606, a91d2c9bdf114d3bb4390d2709b4cb73
I20260619 16:58:51.786478 28818 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 16:58:51.847393 28823 raft_consensus.cc:482] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a91d2c9bdf114d3bb4390d2709b4cb73)
I20260619 16:58:51.847527 28823 raft_consensus.cc:504] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: false } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: false } }
I20260619 16:58:51.847792 28823 leader_election.cc:288] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a91d2c9bdf114d3bb4390d2709b4cb73 (127.20.231.190:36495), 24b2f725bfee4806b3880e8d28ac3b78 (127.20.231.188:46683)
I20260619 16:58:51.848217 28824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.847943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36495 (local address 127.0.0.1:43180)
0619 16:58:51.848083 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:51.848154 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:36495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":87,"threads_started":1}
W20260619 16:58:51.848382 28084 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.231.190:36495: connect: Connection refused (error 111)
I20260619 16:58:51.848929 28824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.848820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36495 (local address 127.0.0.1:43186)
0619 16:58:51.848869 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:51.848903 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:36495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 16:58:51.849090 28084 leader_election.cc:334] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer a91d2c9bdf114d3bb4390d2709b4cb73 (127.20.231.190:36495): Network error: Client connection negotiation failed: client connection to 127.20.231.190:36495: connect: Connection refused (error 111)
I20260619 16:58:51.851214 28825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.847994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:46683 (local address 127.0.0.1:50960)
0619 16:58:51.848108 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:51.848120 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:51.848209 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:51.848529 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:51.848531 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:51.848534 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:51.848594 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:51.848602 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:51.849506 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:51.849507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:51.850280 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:51.850282 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:51.850938 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:51.850940 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:51.851004 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:51.851008 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:58:51.851065 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 16:58:51.851134 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":65,"threads_started":1}
I20260619 16:58:51.851294 28826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.848154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50960 (local address 127.20.231.188:46683)
0619 16:58:51.848367 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:51.848370 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:51.848378 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:58:51.848406 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:51.848407 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:51.848419 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:51.848493 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:51.848696 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:51.849425 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:51.850373 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:51.850807 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:51.850897 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:51.850908 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:51.851150 (+ 242us) server_negotiation.cc:299] Negotiation successful
0619 16:58:51.851204 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":88,"threads_started":1}
I20260619 16:58:51.851498 28305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "24b2f725bfee4806b3880e8d28ac3b78" is_pre_election: true
I20260619 16:58:51.851611 28305 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 674b2e8dfe9a4a8d82791cd75c0e0606 in term 1.
I20260619 16:58:51.851824 28086 leader_election.cc:302] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24b2f725bfee4806b3880e8d28ac3b78, 674b2e8dfe9a4a8d82791cd75c0e0606; no voters: a91d2c9bdf114d3bb4390d2709b4cb73
I20260619 16:58:51.851933 28823 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 16:58:51.851974 28823 raft_consensus.cc:482] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a91d2c9bdf114d3bb4390d2709b4cb73)
I20260619 16:58:51.852000 28823 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [term 1 FOLLOWER]: Advancing to term 2
I20260619 16:58:51.852708 28823 raft_consensus.cc:504] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: false } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: false } }
I20260619 16:58:51.852844 28823 leader_election.cc:288] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [CANDIDATE]: Term 2 election: Requested vote from peers a91d2c9bdf114d3bb4390d2709b4cb73 (127.20.231.190:36495), 24b2f725bfee4806b3880e8d28ac3b78 (127.20.231.188:46683)
I20260619 16:58:51.853003 28825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.852915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36495 (local address 127.0.0.1:43192)
0619 16:58:51.852956 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:51.852981 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:36495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 16:58:51.852998 28305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "24b2f725bfee4806b3880e8d28ac3b78"
I20260619 16:58:51.853091 28305 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [term 1 FOLLOWER]: Advancing to term 2
I20260619 16:58:51.853350 28825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.853233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36495 (local address 127.0.0.1:43202)
0619 16:58:51.853307 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:51.853334 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:36495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260619 16:58:51.853468 28084 leader_election.cc:334] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer a91d2c9bdf114d3bb4390d2709b4cb73 (127.20.231.190:36495): Network error: Client connection negotiation failed: client connection to 127.20.231.190:36495: connect: Connection refused (error 111)
I20260619 16:58:51.853986 28305 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 674b2e8dfe9a4a8d82791cd75c0e0606 in term 2.
I20260619 16:58:51.854139 28086 leader_election.cc:302] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 24b2f725bfee4806b3880e8d28ac3b78, 674b2e8dfe9a4a8d82791cd75c0e0606; no voters: a91d2c9bdf114d3bb4390d2709b4cb73
I20260619 16:58:51.854240 28823 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 16:58:51.854405 28823 raft_consensus.cc:686] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [term 2 LEADER]: Becoming Leader. State: Replica: 674b2e8dfe9a4a8d82791cd75c0e0606, State: Running, Role: LEADER
I20260619 16:58:51.854516 28823 consensus_queue.cc:237] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 11, Committed index: 11, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: false } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: false } }
I20260619 16:58:51.854893 28828 sys_catalog.cc:455] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 674b2e8dfe9a4a8d82791cd75c0e0606. Latest consensus state: current_term: 2 leader_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: false } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: NON_VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: true } } } pending_config { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: false } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: false } } }
I20260619 16:58:51.855008 28828 sys_catalog.cc:458] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:51.855207 28830 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:58:51.855607 28830 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:58:51.855695 28830 catalog_manager.cc:1259] Loaded cluster ID: 1ee2332bf6294effb4cd869600b75170
I20260619 16:58:51.855726 28830 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:58:51.855890 28830 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:58:51.855984 28830 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606: Loaded TSK: 0
I20260619 16:58:51.856118 28830 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:58:51.908248 28099 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50894:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:58:51.910084 28099 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:58:51.911198 28825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.911095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36495 (local address 127.0.0.1:43228)
0619 16:58:51.911132 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:51.911167 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:36495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260619 16:58:51.911262 28305 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [term 2 FOLLOWER]: Refusing update from remote peer 674b2e8dfe9a4a8d82791cd75c0e0606: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260619 16:58:51.911507 28825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.911423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36495 (local address 127.0.0.1:43232)
0619 16:58:51.911465 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:51.911484 (+ 19us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:36495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 16:58:51.911592 28084 consensus_peers.cc:487] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 -> Peer a91d2c9bdf114d3bb4390d2709b4cb73 (127.20.231.190:36495): Couldn't send request to peer a91d2c9bdf114d3bb4390d2709b4cb73. Status: Network error: Client connection negotiation failed: client connection to 127.20.231.190:36495: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 16:58:51.911607 28828 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 11, Time since last communication: 0.000s
I20260619 16:58:51.912369 28818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 674b2e8dfe9a4a8d82791cd75c0e0606. Latest consensus state: current_term: 2 leader_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: false } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: NON_VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: true } } } pending_config { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: false } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: false } } }
I20260619 16:58:51.912460 28818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 16:58:51.912729 28823 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [term 2 LEADER]: Committing config change with OpId 1.12: config changed from index 7 to 12, 24b2f725bfee4806b3880e8d28ac3b78 (127.20.231.188) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: false } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: false } } }
I20260619 16:58:51.912937 28305 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [term 2 FOLLOWER]: Committing config change with OpId 1.12: config changed from index 7 to 12, 24b2f725bfee4806b3880e8d28ac3b78 (127.20.231.188) changed from NON_VOTER to VOTER. New config: { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: false } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: false } } }
I20260619 16:58:51.913691 28828 sys_catalog.cc:455] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" committed_config { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: false } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: false } } }
I20260619 16:58:51.913718 28835 sys_catalog.cc:455] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" committed_config { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: false } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: false } } }
I20260619 16:58:51.913823 28828 sys_catalog.cc:458] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:51.913832 28835 sys_catalog.cc:458] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606 [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:51.914109 28836 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" committed_config { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: false } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: false } } }
I20260619 16:58:51.914192 28836 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 16:58:51.914155 28831 mvcc.cc:205] Tried to move back new op lower bound from 7298614607505973248 to 7298614607276703744. Current Snapshot: MvccSnapshot[applied={T|T < 7298614607505973248 or (T in {7298614607505973248})}]
I20260619 16:58:51.914247 28818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" committed_config { opid_index: 12 OBSOLETE_local: true peers { permanent_uuid: "a91d2c9bdf114d3bb4390d2709b4cb73" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36495 } } peers { permanent_uuid: "674b2e8dfe9a4a8d82791cd75c0e0606" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 44821 } attrs { promote: false } } peers { permanent_uuid: "24b2f725bfee4806b3880e8d28ac3b78" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 46683 } attrs { promote: false } } }
I20260619 16:58:51.914326 28818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 24b2f725bfee4806b3880e8d28ac3b78 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 16:58:51.919241 28825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.915963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:34667 (local address 127.0.0.1:55876)
0619 16:58:51.916002 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:51.916013 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:51.916050 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:51.916494 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:51.916495 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:51.916496 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:51.916547 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:51.916549 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:51.917261 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:51.917262 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:51.918142 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:51.918144 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:51.919015 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:51.919016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:51.919073 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:51.919079 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 16:58:51.919133 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 16:58:51.919173 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260619 16:58:51.919284 28840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.916135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55876 (local address 127.20.231.130:34667)
0619 16:58:51.916257 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:51.916259 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:51.916270 (+ 11us) server_negotiation.cc:407] Connection header received
0619 16:58:51.916302 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:51.916303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:51.916318 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:51.916431 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:51.916631 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:51.917179 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:51.918247 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:51.918921 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:51.919028 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:51.919040 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:51.919153 (+ 113us) server_negotiation.cc:299] Negotiation successful
0619 16:58:51.919202 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":57,"threads_started":1}
I20260619 16:58:51.919516 28546 tablet_service.cc:1467] Processing CreateTablet for tablet 7816608ed65340b798368d8b8f66e47d (DEFAULT_TABLE table=test-table [id=1fa9eaadded64deea9da6861f9d4d957]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:51.919657 28546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7816608ed65340b798368d8b8f66e47d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:51.920094 28824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.916025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:36425 (local address 127.0.0.1:50714)
0619 16:58:51.916080 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:51.916091 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:51.916156 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:51.916701 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:51.916701 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:51.916705 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:51.916756 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:51.916758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:51.917775 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:51.917776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:51.918803 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:51.918805 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:51.919896 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:51.919897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:51.919963 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:51.919966 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:58:51.919995 (+ 29us) client_negotiation.cc:240] Negotiation successful
0619 16:58:51.920033 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":22}
I20260619 16:58:51.920293 28841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.916280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49802 (local address 127.20.231.131:36771)
0619 16:58:51.916407 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:51.916409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:51.916418 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:58:51.916446 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:51.916446 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:51.916458 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:51.916531 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:51.916738 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:51.917583 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:51.919120 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:51.919762 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:51.919860 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:51.919870 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:51.920050 (+ 180us) server_negotiation.cc:299] Negotiation successful
0619 16:58:51.920110 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":65,"threads_started":1}
I20260619 16:58:51.920367 28839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.916086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50714 (local address 127.20.231.129:36425)
0619 16:58:51.916519 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:51.916521 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:51.916531 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:58:51.916559 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:51.916559 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:51.916571 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:51.916834 (+ 263us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:51.916876 (+ 42us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:51.917689 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:51.918914 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:51.919797 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:51.920216 (+ 419us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:51.920226 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:51.920250 (+ 24us) server_negotiation.cc:299] Negotiation successful
0619 16:58:51.920293 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":91,"threads_started":1}
I20260619 16:58:51.920576 28410 tablet_service.cc:1467] Processing CreateTablet for tablet 7816608ed65340b798368d8b8f66e47d (DEFAULT_TABLE table=test-table [id=1fa9eaadded64deea9da6861f9d4d957]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:51.920720 28410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7816608ed65340b798368d8b8f66e47d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:51.920737 28838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.916079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:36771 (local address 127.0.0.1:49802)
0619 16:58:51.916247 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:51.916256 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:51.916330 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:51.916586 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:51.916587 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:51.916589 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:51.916639 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:51.916641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:51.917700 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:51.917701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:51.918998 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:51.919001 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:51.919860 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:51.919862 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:51.919941 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:51.919945 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:58:51.920041 (+ 96us) client_negotiation.cc:240] Negotiation successful
0619 16:58:51.920090 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":35,"thread_start_us":91,"threads_started":1}
I20260619 16:58:51.921195 28842 tablet_bootstrap.cc:492] T 7816608ed65340b798368d8b8f66e47d P 3f0c706426784b9189a3f41b0340d499: Bootstrap starting.
I20260619 16:58:51.921224 28682 tablet_service.cc:1467] Processing CreateTablet for tablet 7816608ed65340b798368d8b8f66e47d (DEFAULT_TABLE table=test-table [id=1fa9eaadded64deea9da6861f9d4d957]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:51.921346 28682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7816608ed65340b798368d8b8f66e47d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:51.921566 28842 tablet_bootstrap.cc:654] T 7816608ed65340b798368d8b8f66e47d P 3f0c706426784b9189a3f41b0340d499: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:51.922084 28842 tablet_bootstrap.cc:492] T 7816608ed65340b798368d8b8f66e47d P 3f0c706426784b9189a3f41b0340d499: No bootstrap required, opened a new log
I20260619 16:58:51.922118 28842 ts_tablet_manager.cc:1397] T 7816608ed65340b798368d8b8f66e47d P 3f0c706426784b9189a3f41b0340d499: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:51.922212 28842 raft_consensus.cc:348] T 7816608ed65340b798368d8b8f66e47d P 3f0c706426784b9189a3f41b0340d499 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c706426784b9189a3f41b0340d499" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34667 } } peers { permanent_uuid: "46441ec526ae4a0694fe9f7907de8b8e" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 36425 } } peers { permanent_uuid: "258ee1befa62410987844618763d8c16" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36771 } }
I20260619 16:58:51.922258 28842 raft_consensus.cc:374] T 7816608ed65340b798368d8b8f66e47d P 3f0c706426784b9189a3f41b0340d499 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:51.922271 28842 raft_consensus.cc:729] T 7816608ed65340b798368d8b8f66e47d P 3f0c706426784b9189a3f41b0340d499 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f0c706426784b9189a3f41b0340d499, State: Initialized, Role: FOLLOWER
I20260619 16:58:51.922341 28842 consensus_queue.cc:260] T 7816608ed65340b798368d8b8f66e47d P 3f0c706426784b9189a3f41b0340d499 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c706426784b9189a3f41b0340d499" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34667 } } peers { permanent_uuid: "46441ec526ae4a0694fe9f7907de8b8e" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 36425 } } peers { permanent_uuid: "258ee1befa62410987844618763d8c16" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36771 } }
I20260619 16:58:51.922406 28844 tablet_bootstrap.cc:492] T 7816608ed65340b798368d8b8f66e47d P 46441ec526ae4a0694fe9f7907de8b8e: Bootstrap starting.
I20260619 16:58:51.922488 28842 ts_tablet_manager.cc:1428] T 7816608ed65340b798368d8b8f66e47d P 3f0c706426784b9189a3f41b0340d499: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:51.922752 28844 tablet_bootstrap.cc:654] T 7816608ed65340b798368d8b8f66e47d P 46441ec526ae4a0694fe9f7907de8b8e: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:51.923102 28847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.922892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36495 (local address 127.20.231.130:36673)
0619 16:58:51.923022 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:51.923064 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:36495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":89,"threads_started":1}
I20260619 16:58:51.923303 28844 tablet_bootstrap.cc:492] T 7816608ed65340b798368d8b8f66e47d P 46441ec526ae4a0694fe9f7907de8b8e: No bootstrap required, opened a new log
I20260619 16:58:51.923344 28844 ts_tablet_manager.cc:1397] T 7816608ed65340b798368d8b8f66e47d P 46441ec526ae4a0694fe9f7907de8b8e: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:51.923470 28844 raft_consensus.cc:348] T 7816608ed65340b798368d8b8f66e47d P 46441ec526ae4a0694fe9f7907de8b8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c706426784b9189a3f41b0340d499" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34667 } } peers { permanent_uuid: "46441ec526ae4a0694fe9f7907de8b8e" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 36425 } } peers { permanent_uuid: "258ee1befa62410987844618763d8c16" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36771 } }
I20260619 16:58:51.923544 28844 raft_consensus.cc:374] T 7816608ed65340b798368d8b8f66e47d P 46441ec526ae4a0694fe9f7907de8b8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:51.923571 28844 raft_consensus.cc:729] T 7816608ed65340b798368d8b8f66e47d P 46441ec526ae4a0694fe9f7907de8b8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46441ec526ae4a0694fe9f7907de8b8e, State: Initialized, Role: FOLLOWER
I20260619 16:58:51.923564 28614 heartbeater.cc:503] Master 127.20.231.189:44821 was elected leader, sending a full tablet report...
I20260619 16:58:51.923619 28844 consensus_queue.cc:260] T 7816608ed65340b798368d8b8f66e47d P 46441ec526ae4a0694fe9f7907de8b8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c706426784b9189a3f41b0340d499" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34667 } } peers { permanent_uuid: "46441ec526ae4a0694fe9f7907de8b8e" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 36425 } } peers { permanent_uuid: "258ee1befa62410987844618763d8c16" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36771 } }
I20260619 16:58:51.923784 28844 ts_tablet_manager.cc:1428] T 7816608ed65340b798368d8b8f66e47d P 46441ec526ae4a0694fe9f7907de8b8e: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 16:58:51.924336 28850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.923995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36495 (local address 127.20.231.129:48021)
0619 16:58:51.924243 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:51.924281 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:36495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":130,"threads_started":1}
I20260619 16:58:51.924351 28478 heartbeater.cc:503] Master 127.20.231.189:44821 was elected leader, sending a full tablet report...
I20260619 16:58:51.924666 28847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.923458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36495 (local address 127.20.231.130:54039)
0619 16:58:51.924601 (+ 1143us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:51.924638 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:36495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1123}
I20260619 16:58:51.924867 28846 tablet_bootstrap.cc:492] T 7816608ed65340b798368d8b8f66e47d P 258ee1befa62410987844618763d8c16: Bootstrap starting.
I20260619 16:58:51.925264 28846 tablet_bootstrap.cc:654] T 7816608ed65340b798368d8b8f66e47d P 258ee1befa62410987844618763d8c16: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:51.925817 28846 tablet_bootstrap.cc:492] T 7816608ed65340b798368d8b8f66e47d P 258ee1befa62410987844618763d8c16: No bootstrap required, opened a new log
I20260619 16:58:51.925889 28846 ts_tablet_manager.cc:1397] T 7816608ed65340b798368d8b8f66e47d P 258ee1befa62410987844618763d8c16: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:51.926038 28846 raft_consensus.cc:348] T 7816608ed65340b798368d8b8f66e47d P 258ee1befa62410987844618763d8c16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c706426784b9189a3f41b0340d499" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34667 } } peers { permanent_uuid: "46441ec526ae4a0694fe9f7907de8b8e" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 36425 } } peers { permanent_uuid: "258ee1befa62410987844618763d8c16" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36771 } }
I20260619 16:58:51.926129 28846 raft_consensus.cc:374] T 7816608ed65340b798368d8b8f66e47d P 258ee1befa62410987844618763d8c16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:51.926169 28846 raft_consensus.cc:729] T 7816608ed65340b798368d8b8f66e47d P 258ee1befa62410987844618763d8c16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 258ee1befa62410987844618763d8c16, State: Initialized, Role: FOLLOWER
I20260619 16:58:51.926234 28846 consensus_queue.cc:260] T 7816608ed65340b798368d8b8f66e47d P 258ee1befa62410987844618763d8c16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c706426784b9189a3f41b0340d499" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34667 } } peers { permanent_uuid: "46441ec526ae4a0694fe9f7907de8b8e" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 36425 } } peers { permanent_uuid: "258ee1befa62410987844618763d8c16" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36771 } }
I20260619 16:58:51.926409 28846 ts_tablet_manager.cc:1428] T 7816608ed65340b798368d8b8f66e47d P 258ee1befa62410987844618763d8c16: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 16:58:51.926762 28853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.926551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36495 (local address 127.20.231.131:47953)
0619 16:58:51.926692 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:51.926725 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:36495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":75,"threads_started":1}
I20260619 16:58:51.927057 28750 heartbeater.cc:503] Master 127.20.231.189:44821 was elected leader, sending a full tablet report...
I20260619 16:58:51.927171 28853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.926992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36495 (local address 127.20.231.131:51201)
0619 16:58:51.927114 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:51.927139 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:36495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":74,"threads_started":1}
I20260619 16:58:51.938933 28845 raft_consensus.cc:482] T 7816608ed65340b798368d8b8f66e47d P 3f0c706426784b9189a3f41b0340d499 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:58:51.939015 28845 raft_consensus.cc:504] T 7816608ed65340b798368d8b8f66e47d P 3f0c706426784b9189a3f41b0340d499 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c706426784b9189a3f41b0340d499" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34667 } } peers { permanent_uuid: "46441ec526ae4a0694fe9f7907de8b8e" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 36425 } } peers { permanent_uuid: "258ee1befa62410987844618763d8c16" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36771 } }
I20260619 16:58:51.939291 28845 leader_election.cc:288] T 7816608ed65340b798368d8b8f66e47d P 3f0c706426784b9189a3f41b0340d499 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 46441ec526ae4a0694fe9f7907de8b8e (127.20.231.129:36425), 258ee1befa62410987844618763d8c16 (127.20.231.131:36771)
I20260619 16:58:51.942332 28847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.939458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:36425 (local address 127.20.231.130:50375)
0619 16:58:51.939506 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:51.939521 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:51.939574 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:51.939742 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:51.939743 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:51.939746 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:51.939830 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:51.939833 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:51.940495 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:51.940496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:51.941389 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:51.941391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:51.942091 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:51.942092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:51.942184 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:51.942188 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:58:51.942236 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 16:58:51.942268 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260619 16:58:51.942361 28839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.939575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:50375 (local address 127.20.231.129:36425)
0619 16:58:51.939619 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:51.939621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:51.939628 (+ 7us) server_negotiation.cc:407] Connection header received
0619 16:58:51.939658 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:51.939659 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:51.939666 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:51.939716 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:51.939905 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:51.940415 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:51.941470 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:51.942010 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:51.942086 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:51.942096 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:51.942254 (+ 158us) server_negotiation.cc:299] Negotiation successful
0619 16:58:51.942295 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 16:58:51.942776 28430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7816608ed65340b798368d8b8f66e47d" candidate_uuid: "3f0c706426784b9189a3f41b0340d499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46441ec526ae4a0694fe9f7907de8b8e" is_pre_election: true
I20260619 16:58:51.942917 28430 raft_consensus.cc:2454] T 7816608ed65340b798368d8b8f66e47d P 46441ec526ae4a0694fe9f7907de8b8e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f0c706426784b9189a3f41b0340d499 in term 0.
I20260619 16:58:51.943084 28501 leader_election.cc:302] T 7816608ed65340b798368d8b8f66e47d P 3f0c706426784b9189a3f41b0340d499 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f0c706426784b9189a3f41b0340d499, 46441ec526ae4a0694fe9f7907de8b8e; no voters:
I20260619 16:58:51.943219 28845 raft_consensus.cc:2781] T 7816608ed65340b798368d8b8f66e47d P 3f0c706426784b9189a3f41b0340d499 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:58:51.943264 28845 raft_consensus.cc:482] T 7816608ed65340b798368d8b8f66e47d P 3f0c706426784b9189a3f41b0340d499 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:58:51.943284 28845 raft_consensus.cc:3037] T 7816608ed65340b798368d8b8f66e47d P 3f0c706426784b9189a3f41b0340d499 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:51.943482 28855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.939458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:36771 (local address 127.20.231.130:52843)
0619 16:58:51.939827 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:51.939836 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:51.939902 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:51.940588 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:51.940589 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:51.940591 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:51.940677 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:51.940678 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:51.941277 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:51.941278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:51.942349 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:51.942351 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:51.943227 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:51.943228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:51.943307 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:51.943311 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:58:51.943377 (+ 66us) client_negotiation.cc:240] Negotiation successful
0619 16:58:51.943420 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"mutex_wait_us":75,"thread_start_us":258,"threads_started":1}
I20260619 16:58:51.943516 28841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.939513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:52843 (local address 127.20.231.131:36771)
0619 16:58:51.939575 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:51.939576 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:51.940486 (+ 910us) server_negotiation.cc:407] Connection header received
0619 16:58:51.940512 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:51.940512 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:51.940521 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:51.940584 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:51.940757 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:51.941202 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:51.942451 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:51.943083 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:51.943153 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:51.943163 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:51.943398 (+ 235us) server_negotiation.cc:299] Negotiation successful
0619 16:58:51.943446 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 16:58:51.943727 28702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7816608ed65340b798368d8b8f66e47d" candidate_uuid: "3f0c706426784b9189a3f41b0340d499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "258ee1befa62410987844618763d8c16" is_pre_election: true
I20260619 16:58:51.943861 28702 raft_consensus.cc:2454] T 7816608ed65340b798368d8b8f66e47d P 258ee1befa62410987844618763d8c16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3f0c706426784b9189a3f41b0340d499 in term 0.
I20260619 16:58:51.944244 28845 raft_consensus.cc:504] T 7816608ed65340b798368d8b8f66e47d P 3f0c706426784b9189a3f41b0340d499 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c706426784b9189a3f41b0340d499" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34667 } } peers { permanent_uuid: "46441ec526ae4a0694fe9f7907de8b8e" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 36425 } } peers { permanent_uuid: "258ee1befa62410987844618763d8c16" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36771 } }
I20260619 16:58:51.944389 28845 leader_election.cc:288] T 7816608ed65340b798368d8b8f66e47d P 3f0c706426784b9189a3f41b0340d499 [CANDIDATE]: Term 1 election: Requested vote from peers 46441ec526ae4a0694fe9f7907de8b8e (127.20.231.129:36425), 258ee1befa62410987844618763d8c16 (127.20.231.131:36771)
I20260619 16:58:51.944527 28430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7816608ed65340b798368d8b8f66e47d" candidate_uuid: "3f0c706426784b9189a3f41b0340d499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46441ec526ae4a0694fe9f7907de8b8e"
I20260619 16:58:51.944552 28702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7816608ed65340b798368d8b8f66e47d" candidate_uuid: "3f0c706426784b9189a3f41b0340d499" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "258ee1befa62410987844618763d8c16"
I20260619 16:58:51.944617 28702 raft_consensus.cc:3037] T 7816608ed65340b798368d8b8f66e47d P 258ee1befa62410987844618763d8c16 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:51.944610 28430 raft_consensus.cc:3037] T 7816608ed65340b798368d8b8f66e47d P 46441ec526ae4a0694fe9f7907de8b8e [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:51.945411 28430 raft_consensus.cc:2454] T 7816608ed65340b798368d8b8f66e47d P 46441ec526ae4a0694fe9f7907de8b8e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f0c706426784b9189a3f41b0340d499 in term 1.
I20260619 16:58:51.945413 28702 raft_consensus.cc:2454] T 7816608ed65340b798368d8b8f66e47d P 258ee1befa62410987844618763d8c16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3f0c706426784b9189a3f41b0340d499 in term 1.
I20260619 16:58:51.945576 28501 leader_election.cc:302] T 7816608ed65340b798368d8b8f66e47d P 3f0c706426784b9189a3f41b0340d499 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3f0c706426784b9189a3f41b0340d499, 46441ec526ae4a0694fe9f7907de8b8e; no voters:
I20260619 16:58:51.945664 28845 raft_consensus.cc:2781] T 7816608ed65340b798368d8b8f66e47d P 3f0c706426784b9189a3f41b0340d499 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:51.945833 28845 raft_consensus.cc:686] T 7816608ed65340b798368d8b8f66e47d P 3f0c706426784b9189a3f41b0340d499 [term 1 LEADER]: Becoming Leader. State: Replica: 3f0c706426784b9189a3f41b0340d499, State: Running, Role: LEADER
I20260619 16:58:51.945910 28845 consensus_queue.cc:237] T 7816608ed65340b798368d8b8f66e47d P 3f0c706426784b9189a3f41b0340d499 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c706426784b9189a3f41b0340d499" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34667 } } peers { permanent_uuid: "46441ec526ae4a0694fe9f7907de8b8e" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 36425 } } peers { permanent_uuid: "258ee1befa62410987844618763d8c16" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36771 } }
I20260619 16:58:51.946587 28099 catalog_manager.cc:5495] T 7816608ed65340b798368d8b8f66e47d P 3f0c706426784b9189a3f41b0340d499 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3f0c706426784b9189a3f41b0340d499 (127.20.231.130). New cstate: current_term: 1 leader_uuid: "3f0c706426784b9189a3f41b0340d499" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3f0c706426784b9189a3f41b0340d499" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34667 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "46441ec526ae4a0694fe9f7907de8b8e" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 36425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "258ee1befa62410987844618763d8c16" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36771 } health_report { overall_health: UNKNOWN } } }
I20260619 16:58:51.947118 28855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.946999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36495 (local address 127.20.231.130:49647)
0619 16:58:51.947050 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:51.947091 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:36495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 16:58:51.947413 28479 tablet.cc:2369] T 7816608ed65340b798368d8b8f66e47d P 46441ec526ae4a0694fe9f7907de8b8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:58:51.955005 28840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.952599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55892 (local address 127.20.231.130:34667)
0619 16:58:51.952640 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:51.952642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:51.952651 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:58:51.952685 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:51.952685 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:51.952706 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:51.952802 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:58:51.952887 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:51.953369 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:51.954185 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:51.954404 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:51.954474 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:51.954582 (+ 108us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:58:51.954756 (+ 174us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:58:51.954818 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:51.954923 (+ 105us) server_negotiation.cc:299] Negotiation successful
0619 16:58:51.954945 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 16:58:51.956099 28430 raft_consensus.cc:1261] T 7816608ed65340b798368d8b8f66e47d P 46441ec526ae4a0694fe9f7907de8b8e [term 1 FOLLOWER]: Refusing update from remote peer 3f0c706426784b9189a3f41b0340d499: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:51.956180 28702 raft_consensus.cc:1261] T 7816608ed65340b798368d8b8f66e47d P 258ee1befa62410987844618763d8c16 [term 1 FOLLOWER]: Refusing update from remote peer 3f0c706426784b9189a3f41b0340d499: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:51.956377 28845 consensus_queue.cc:1035] T 7816608ed65340b798368d8b8f66e47d P 3f0c706426784b9189a3f41b0340d499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46441ec526ae4a0694fe9f7907de8b8e" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 36425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:51.956502 28856 consensus_queue.cc:1035] T 7816608ed65340b798368d8b8f66e47d P 3f0c706426784b9189a3f41b0340d499 [LEADER]: Connected to new peer: Peer: permanent_uuid: "258ee1befa62410987844618763d8c16" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36771 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:51.957296 28850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.957175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36495 (local address 127.20.231.129:56789)
0619 16:58:51.957223 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:51.957264 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:36495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260619 16:58:51.958460 28854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.958316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36495 (local address 127.20.231.131:52951)
0619 16:58:51.958378 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:51.958422 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:36495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 16:58:51.958966 28850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.958875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36495 (local address 127.20.231.129:49219)
0619 16:58:51.958910 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:51.958937 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:36495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260619 16:58:51.959169 28860 mvcc.cc:205] Tried to move back new op lower bound from 7298614607690117120 to 7298614607650955264. Current Snapshot: MvccSnapshot[applied={T|T < 7298614607690117120 or (T in {7298614607690117120})}]
I20260619 16:58:51.959417 28855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.959324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36495 (local address 127.20.231.130:49277)
0619 16:58:51.959361 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:51.959388 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:36495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260619 16:58:51.960232 28855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.960123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36495 (local address 127.20.231.130:36551)
0619 16:58:51.960157 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:51.960199 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:36495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260619 16:58:51.960762 28854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.960522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36495 (local address 127.20.231.131:40441)
0619 16:58:51.960704 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:51.960733 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:36495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166}
I20260619 16:58:51.961791 28100 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:58:51.961894 28100 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50894:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:58:51.963263 28100 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 674b2e8dfe9a4a8d82791cd75c0e0606: Sending DeleteTablet for 3 replicas of tablet 7816608ed65340b798368d8b8f66e47d
I20260619 16:58:51.963536 28410 tablet_service.cc:1514] Processing DeleteTablet for tablet 7816608ed65340b798368d8b8f66e47d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:51 UTC) from {username='oryx', principal='oryx/127.20.231.189@KRBTEST.COM'} at 127.0.0.1:50714
I20260619 16:58:51.963584 28546 tablet_service.cc:1514] Processing DeleteTablet for tablet 7816608ed65340b798368d8b8f66e47d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:51 UTC) from {username='oryx', principal='oryx/127.20.231.189@KRBTEST.COM'} at 127.0.0.1:55876
I20260619 16:58:51.963762 28869 tablet_replica.cc:330] T 7816608ed65340b798368d8b8f66e47d P 46441ec526ae4a0694fe9f7907de8b8e: stopping tablet replica
I20260619 16:58:51.963820 28869 raft_consensus.cc:2229] T 7816608ed65340b798368d8b8f66e47d P 46441ec526ae4a0694fe9f7907de8b8e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:58:51.963878 28869 raft_consensus.cc:2258] T 7816608ed65340b798368d8b8f66e47d P 46441ec526ae4a0694fe9f7907de8b8e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:51.963984 28682 tablet_service.cc:1514] Processing DeleteTablet for tablet 7816608ed65340b798368d8b8f66e47d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:51 UTC) from {username='oryx', principal='oryx/127.20.231.189@KRBTEST.COM'} at 127.0.0.1:49802
I20260619 16:58:51.964155 28871 tablet_replica.cc:330] T 7816608ed65340b798368d8b8f66e47d P 258ee1befa62410987844618763d8c16: stopping tablet replica
I20260619 16:58:51.964241 28869 ts_tablet_manager.cc:1905] T 7816608ed65340b798368d8b8f66e47d P 46441ec526ae4a0694fe9f7907de8b8e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:51.964270 28871 raft_consensus.cc:2229] T 7816608ed65340b798368d8b8f66e47d P 258ee1befa62410987844618763d8c16 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:58:51.964356 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 28347
I20260619 16:58:51.964450 28871 raft_consensus.cc:2258] T 7816608ed65340b798368d8b8f66e47d P 258ee1befa62410987844618763d8c16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:51.965205 28871 ts_tablet_manager.cc:1905] T 7816608ed65340b798368d8b8f66e47d P 258ee1befa62410987844618763d8c16: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:51.967087 28871 ts_tablet_manager.cc:1918] T 7816608ed65340b798368d8b8f66e47d P 258ee1befa62410987844618763d8c16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:58:51.967168 28871 log.cc:1196] T 7816608ed65340b798368d8b8f66e47d P 258ee1befa62410987844618763d8c16: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781888301345144-21406-0/minicluster-data/ts-2/wal/wals/7816608ed65340b798368d8b8f66e47d
I20260619 16:58:51.967447 28871 ts_tablet_manager.cc:1939] T 7816608ed65340b798368d8b8f66e47d P 258ee1befa62410987844618763d8c16: Deleting consensus metadata
I20260619 16:58:51.968156 28084 catalog_manager.cc:4841] TS 258ee1befa62410987844618763d8c16 (127.20.231.131:36771): tablet 7816608ed65340b798368d8b8f66e47d (table test-table [id=1fa9eaadded64deea9da6861f9d4d957]) successfully deleted
I20260619 16:58:51.971251 28870 tablet_replica.cc:330] T 7816608ed65340b798368d8b8f66e47d P 3f0c706426784b9189a3f41b0340d499: stopping tablet replica
I20260619 16:58:51.971308 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 28483
I20260619 16:58:51.971328 28870 raft_consensus.cc:2229] T 7816608ed65340b798368d8b8f66e47d P 3f0c706426784b9189a3f41b0340d499 [term 1 LEADER]: Raft consensus shutting down.
I20260619 16:58:51.971339 28838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:51.971201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:36425 (local address 127.0.0.1:50728)
0619 16:58:51.971261 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:51.971294 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.129:36425: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260619 16:58:51.972570 28086 catalog_manager.cc:4584] TS 46441ec526ae4a0694fe9f7907de8b8e (127.20.231.129:36425): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7816608ed65340b798368d8b8f66e47d: Network error: Client connection negotiation failed: client connection to 127.20.231.129:36425: connect: Connection refused (error 111)
W20260619 16:58:51.977638 28838 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:51.977419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:34667 (local address 127.0.0.1:55908)
0619 16:58:51.977475 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:51.977486 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:51.977535 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:51.977585 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.130:34667: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 16:58:51.977809 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 28619
I20260619 16:58:51.983081 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 27967
I20260619 16:58:51.988149 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 28154
2026-06-19T16:58:51Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (4574 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:58:52 dist-test-slave-vnbj krb5kdc[28874](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:58:52 dist-test-slave-vnbj krb5kdc[28874](info): set up 2 sockets
Jun 19 16:58:52 dist-test-slave-vnbj krb5kdc[28874](info): commencing operation
krb5kdc: starting...
W20260619 16:58:53.047441 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 16:58:53 dist-test-slave-vnbj krb5kdc[28874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:58:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:58:53Z Disabled control of system clock
WARNING: no policy specified for oryx/127.20.231.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.231.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.231.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.190.keytab.
Entry for principal oryx/127.20.231.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.190.keytab.
I20260619 16:58:53.171514 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:41933
--webserver_interface=127.20.231.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.190.keytab
--principal=oryx/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:35773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:41933
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:53.254638 28887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:53.254904 28887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:53.254927 28887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:53.256212 28887 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:53.256258 28887 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 16:58:53.256271 28887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:53.256284 28887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:53.256315 28887 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:53.256342 28887 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:58:53.257933 28887 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:35773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:41933
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.190.keytab
--principal=oryx/127.20.231.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:41933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:53.258211 28887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:53.258460 28887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:53.263707 28893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:53.263707 28892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:53.263833 28887 server_base.cc:1034] running on GCE node
W20260619 16:58:53.263762 28895 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:53.264345 28887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:53.264662 28887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:53.265839 28887 hybrid_clock.cc:648] HybridClock initialized: now 1781888333265819 us; error 39 us; skew 500 ppm
Jun 19 16:58:53 dist-test-slave-vnbj krb5kdc[28874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888333, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:53.275063 28887 init.cc:377] Logged in from keytab as oryx/127.20.231.190@KRBTEST.COM (short username oryx)
I20260619 16:58:53.275522 28887 webserver.cc:460] Webserver started at http://127.20.231.190:36667/ using document root <none> and password file <none>
I20260619 16:58:53.275732 28887 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:53.275776 28887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:53.275888 28887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:53.277009 28887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "4616d2b9af7648f9ac7f8bcc6a5e185d"
format_stamp: "Formatted at 2026-06-19 16:58:53 on dist-test-slave-vnbj"
server_key: "64034c31443a1aaf6975b0b5a69ac3b6"
server_key_iv: "93198f36704403e629baed483fe0e69e"
server_key_version: "encryptionkey@0"
I20260619 16:58:53.277320 28887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "4616d2b9af7648f9ac7f8bcc6a5e185d"
format_stamp: "Formatted at 2026-06-19 16:58:53 on dist-test-slave-vnbj"
server_key: "64034c31443a1aaf6975b0b5a69ac3b6"
server_key_iv: "93198f36704403e629baed483fe0e69e"
server_key_version: "encryptionkey@0"
I20260619 16:58:53.278657 28887 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:58:53.279548 28902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:53.279743 28887 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:53.279879 28887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "4616d2b9af7648f9ac7f8bcc6a5e185d"
format_stamp: "Formatted at 2026-06-19 16:58:53 on dist-test-slave-vnbj"
server_key: "64034c31443a1aaf6975b0b5a69ac3b6"
server_key_iv: "93198f36704403e629baed483fe0e69e"
server_key_version: "encryptionkey@0"
I20260619 16:58:53.279974 28887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:53.303630 28887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:53.305694 28887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:53.305913 28887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:53.310909 28887 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:41933
I20260619 16:58:53.310988 28964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:41933 every 8 connection(s)
I20260619 16:58:53.311347 28887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 16:58:53.312012 28965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:53.314545 28965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4616d2b9af7648f9ac7f8bcc6a5e185d: Bootstrap starting.
I20260619 16:58:53.315279 28965 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4616d2b9af7648f9ac7f8bcc6a5e185d: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:53.315760 28965 log.cc:824] T 00000000000000000000000000000000 P 4616d2b9af7648f9ac7f8bcc6a5e185d: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:53.316735 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 28887
I20260619 16:58:53.316844 28965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4616d2b9af7648f9ac7f8bcc6a5e185d: No bootstrap required, opened a new log
I20260619 16:58:53.316854 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 16:58:53.317060 21406 external_mini_cluster.cc:1428] Setting key 4e29661b6e103085435f9a9f8cb0e99c
I20260619 16:58:53.319818 28965 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4616d2b9af7648f9ac7f8bcc6a5e185d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4616d2b9af7648f9ac7f8bcc6a5e185d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41933 } }
I20260619 16:58:53.319998 28965 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4616d2b9af7648f9ac7f8bcc6a5e185d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:53.320035 28965 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4616d2b9af7648f9ac7f8bcc6a5e185d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4616d2b9af7648f9ac7f8bcc6a5e185d, State: Initialized, Role: FOLLOWER
I20260619 16:58:53.320222 28965 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4616d2b9af7648f9ac7f8bcc6a5e185d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4616d2b9af7648f9ac7f8bcc6a5e185d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41933 } }
I20260619 16:58:53.320322 28965 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4616d2b9af7648f9ac7f8bcc6a5e185d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:58:53.320369 28965 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4616d2b9af7648f9ac7f8bcc6a5e185d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:58:53.320420 28965 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4616d2b9af7648f9ac7f8bcc6a5e185d [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:53.321115 28965 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4616d2b9af7648f9ac7f8bcc6a5e185d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4616d2b9af7648f9ac7f8bcc6a5e185d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41933 } }
I20260619 16:58:53.321238 28965 leader_election.cc:302] T 00000000000000000000000000000000 P 4616d2b9af7648f9ac7f8bcc6a5e185d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4616d2b9af7648f9ac7f8bcc6a5e185d; no voters:
I20260619 16:58:53.321462 28965 leader_election.cc:288] T 00000000000000000000000000000000 P 4616d2b9af7648f9ac7f8bcc6a5e185d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:58:53.321496 28970 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4616d2b9af7648f9ac7f8bcc6a5e185d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:53.321597 28970 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4616d2b9af7648f9ac7f8bcc6a5e185d [term 1 LEADER]: Becoming Leader. State: Replica: 4616d2b9af7648f9ac7f8bcc6a5e185d, State: Running, Role: LEADER
I20260619 16:58:53.321686 28970 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4616d2b9af7648f9ac7f8bcc6a5e185d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4616d2b9af7648f9ac7f8bcc6a5e185d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41933 } }
I20260619 16:58:53.321768 28965 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4616d2b9af7648f9ac7f8bcc6a5e185d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:53.322109 28970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4616d2b9af7648f9ac7f8bcc6a5e185d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4616d2b9af7648f9ac7f8bcc6a5e185d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4616d2b9af7648f9ac7f8bcc6a5e185d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41933 } } }
I20260619 16:58:53.322189 28970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4616d2b9af7648f9ac7f8bcc6a5e185d [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:53.322302 28970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4616d2b9af7648f9ac7f8bcc6a5e185d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4616d2b9af7648f9ac7f8bcc6a5e185d. Latest consensus state: current_term: 1 leader_uuid: "4616d2b9af7648f9ac7f8bcc6a5e185d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4616d2b9af7648f9ac7f8bcc6a5e185d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41933 } } }
I20260619 16:58:53.322367 28970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4616d2b9af7648f9ac7f8bcc6a5e185d [sys.catalog]: This master's current role is: LEADER
Jun 19 16:58:53 dist-test-slave-vnbj krb5kdc[28874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888333, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM
W20260619 16:58:53.324026 28985 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4616d2b9af7648f9ac7f8bcc6a5e185d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 16:58:53.324864 28985 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 16:58:53.324980 28975 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:58:53.325194 28975 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:58:53.327368 28975 catalog_manager.cc:1347] Generated new cluster ID: 2e32e00e06d34bef907f2cd98337962c
I20260619 16:58:53.327445 28975 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:58:53.329408 28969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:53.317605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49932 (local address 127.20.231.190:41933)
0619 16:58:53.317922 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:53.317931 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:53.317958 (+ 27us) server_negotiation.cc:407] Connection header received
0619 16:58:53.318469 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:53.318477 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:53.318530 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:53.318684 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:53.319396 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.320395 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.321255 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.321905 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.322076 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:53.324733 (+ 2657us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:53.324737 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:53.324738 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:53.324752 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:53.326910 (+ 2158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:53.327375 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:53.327378 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:53.327378 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:53.327484 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:53.327715 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:53.327716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:53.327716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:53.328051 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:53.328683 (+ 632us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:53.328769 (+ 86us) server_negotiation.cc:299] Negotiation successful
0619 16:58:53.328916 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":217,"threads_started":1}
I20260619 16:58:53.333101 28975 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:58:53.333858 28975 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:58:53.343535 28975 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4616d2b9af7648f9ac7f8bcc6a5e185d: Generated new TSK 0
I20260619 16:58:53.343753 28975 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.20.231.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.231.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.231.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.129.keytab.
Entry for principal oryx/127.20.231.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.129.keytab.
I20260619 16:58:53.374083 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.129:0
--local_ip_for_outbound_sockets=127.20.231.129
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:41933
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.129.keytab
--principal=oryx/127.20.231.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:35773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:53.457306 28991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:53.457531 28991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:53.457556 28991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:53.458763 28991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:53.458810 28991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:53.458832 28991 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 16:58:53.458894 28991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.129
I20260619 16:58:53.460454 28991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:35773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.129.keytab
--principal=oryx/127.20.231.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.231.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.231.190:41933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.129
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:53.460640 28991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:53.460822 28991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:53.465853 28999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:53.465931 28996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:53.465860 28997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:53.466148 28991 server_base.cc:1034] running on GCE node
I20260619 16:58:53.466380 28991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:53.466588 28991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:53.467777 28991 hybrid_clock.cc:648] HybridClock initialized: now 1781888333467737 us; error 35 us; skew 500 ppm
Jun 19 16:58:53 dist-test-slave-vnbj krb5kdc[28874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888333, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:53.470288 28991 init.cc:377] Logged in from keytab as oryx/127.20.231.129@KRBTEST.COM (short username oryx)
I20260619 16:58:53.470752 28991 webserver.cc:460] Webserver started at http://127.20.231.129:37473/ using document root <none> and password file <none>
I20260619 16:58:53.471016 28991 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:53.471081 28991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:53.471236 28991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:53.472345 28991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-0/data/instance:
uuid: "5c9cb6210ca6432088b94e62e5b395ac"
format_stamp: "Formatted at 2026-06-19 16:58:53 on dist-test-slave-vnbj"
server_key: "522ced286ce00835c812ba9cf22a9cae"
server_key_iv: "3e7cb3e2d2cb7701fcaa1c8f32660d3d"
server_key_version: "encryptionkey@0"
I20260619 16:58:53.472733 28991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance:
uuid: "5c9cb6210ca6432088b94e62e5b395ac"
format_stamp: "Formatted at 2026-06-19 16:58:53 on dist-test-slave-vnbj"
server_key: "522ced286ce00835c812ba9cf22a9cae"
server_key_iv: "3e7cb3e2d2cb7701fcaa1c8f32660d3d"
server_key_version: "encryptionkey@0"
I20260619 16:58:53.474117 28991 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:58:53.474984 29006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:53.475186 28991 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:53.475286 28991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-0/wal
uuid: "5c9cb6210ca6432088b94e62e5b395ac"
format_stamp: "Formatted at 2026-06-19 16:58:53 on dist-test-slave-vnbj"
server_key: "522ced286ce00835c812ba9cf22a9cae"
server_key_iv: "3e7cb3e2d2cb7701fcaa1c8f32660d3d"
server_key_version: "encryptionkey@0"
I20260619 16:58:53.475375 28991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:53.491467 28991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:53.493643 28991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:53.493831 28991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:53.494388 28991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:53.494453 28991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:53.494498 28991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:53.494525 28991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:53.501163 28991 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.129:46675
I20260619 16:58:53.501307 29121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.129:46675 every 8 connection(s)
I20260619 16:58:53.501579 28991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
Jun 19 16:58:53 dist-test-slave-vnbj krb5kdc[28874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888333, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM
I20260619 16:58:53.505151 28969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:53.494983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:44935 (local address 127.20.231.190:41933)
0619 16:58:53.495053 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:53.495056 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:53.495293 (+ 237us) server_negotiation.cc:407] Connection header received
0619 16:58:53.495880 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:53.495881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:53.495893 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:53.495961 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:53.496865 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.497739 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.499359 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.499731 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.499826 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:53.502641 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:53.502642 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:53.502642 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:53.502650 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:53.504061 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:53.504423 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:53.504424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:53.504424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:53.504473 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:53.504672 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:53.504672 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:53.504672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:53.504774 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:53.504823 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:53.505029 (+ 206us) server_negotiation.cc:299] Negotiation successful
0619 16:58:53.505089 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 16:58:53.505419 29015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:53.494974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41933 (local address 127.20.231.129:44935)
0619 16:58:53.495164 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:53.495206 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:53.495780 (+ 574us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:53.496049 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:53.496054 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:53.496451 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:53.496741 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:53.496748 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.497854 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.497855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:53.499272 (+ 1417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:53.499274 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.499836 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.499837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:53.500013 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:53.500622 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:53.500633 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:53.502545 (+ 1912us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:53.504133 (+ 1588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:53.504138 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:53.504140 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:53.504359 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:53.504525 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:53.504526 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:53.504526 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:53.504623 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:53.504823 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:53.504827 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:53.504939 (+ 112us) client_negotiation.cc:769] Sending connection context
0619 16:58:53.505020 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 16:58:53.505138 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":88,"threads_started":1}
I20260619 16:58:53.506695 29122 heartbeater.cc:347] Connected to a master server at 127.20.231.190:41933
I20260619 16:58:53.506778 29122 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:53.506963 29122 heartbeater.cc:511] Master 127.20.231.190:41933 requested a full tablet report, sending...
I20260619 16:58:53.507409 28918 ts_manager.cc:194] Registered new tserver with Master: 5c9cb6210ca6432088b94e62e5b395ac (127.20.231.129:46675)
I20260619 16:58:53.508029 28918 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.231.129@KRBTEST.COM'} at 127.20.231.129:44935
I20260619 16:58:53.509166 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 28991
I20260619 16:58:53.509312 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance
I20260619 16:58:53.509523 21406 external_mini_cluster.cc:1428] Setting key 7806c70246ca221fe23890b6d800b684
I20260619 16:58:53.512502 28969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:53.506727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:57709 (local address 127.20.231.190:41933)
0619 16:58:53.506781 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:53.506783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:53.506947 (+ 164us) server_negotiation.cc:407] Connection header received
0619 16:58:53.507009 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:53.507010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:53.507020 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:53.507071 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:53.507389 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.508016 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.508746 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.508977 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.509051 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:53.509985 (+ 934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:53.509986 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:53.509986 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:53.509992 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:53.511322 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:53.511733 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:53.511734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:53.511735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:53.511811 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:53.512011 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:53.512012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:53.512012 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:53.512117 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:53.512183 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:53.512339 (+ 156us) server_negotiation.cc:299] Negotiation successful
0619 16:58:53.512412 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":104,"server-negotiator.queue_time_us":32}
I20260619 16:58:53.512514 29128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:53.506694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41933 (local address 127.20.231.129:57709)
0619 16:58:53.506881 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:53.506892 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:53.506950 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:53.507078 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:53.507079 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:53.507253 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:53.507301 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:53.507303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.508103 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.508104 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:53.508660 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:53.508662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.509050 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.509051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:53.509115 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:53.509537 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:53.509542 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:53.509911 (+ 369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:53.511398 (+ 1487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:53.511399 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:53.511399 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:53.511630 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:53.511886 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:53.511887 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:53.511887 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:53.511942 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:53.512185 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:53.512186 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:53.512231 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 16:58:53.512313 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 16:58:53.512413 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":85,"threads_started":1}
WARNING: no policy specified for oryx/127.20.231.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.231.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.231.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.130.keytab.
Entry for principal oryx/127.20.231.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.130.keytab.
I20260619 16:58:53.534122 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.130:0
--local_ip_for_outbound_sockets=127.20.231.130
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:41933
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.130.keytab
--principal=oryx/127.20.231.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:35773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:53.631290 29131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:53.631620 29131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:53.631666 29131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:53.633651 29131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:53.633709 29131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:53.633740 29131 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 16:58:53.633759 29131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.130
I20260619 16:58:53.635802 29131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:35773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.130.keytab
--principal=oryx/127.20.231.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.231.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.231.190:41933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.130
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:53.636122 29131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:53.636417 29131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:53.642609 29136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:53.642675 29137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:53.642660 29139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:53.642975 29131 server_base.cc:1034] running on GCE node
I20260619 16:58:53.643150 29131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:53.643370 29131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:53.644515 29131 hybrid_clock.cc:648] HybridClock initialized: now 1781888333644502 us; error 26 us; skew 500 ppm
Jun 19 16:58:53 dist-test-slave-vnbj krb5kdc[28874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888333, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:53.647089 29131 init.cc:377] Logged in from keytab as oryx/127.20.231.130@KRBTEST.COM (short username oryx)
I20260619 16:58:53.647591 29131 webserver.cc:460] Webserver started at http://127.20.231.130:34193/ using document root <none> and password file <none>
I20260619 16:58:53.647861 29131 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:53.647908 29131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:53.648065 29131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:53.649158 29131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-1/data/instance:
uuid: "464343bddb8a42d39d18253f7ddfc32a"
format_stamp: "Formatted at 2026-06-19 16:58:53 on dist-test-slave-vnbj"
server_key: "f25def7f772e5748bdc9eb4e565243ef"
server_key_iv: "d2a42d126cd36dc92de854a728a3add1"
server_key_version: "encryptionkey@0"
I20260619 16:58:53.649528 29131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance:
uuid: "464343bddb8a42d39d18253f7ddfc32a"
format_stamp: "Formatted at 2026-06-19 16:58:53 on dist-test-slave-vnbj"
server_key: "f25def7f772e5748bdc9eb4e565243ef"
server_key_iv: "d2a42d126cd36dc92de854a728a3add1"
server_key_version: "encryptionkey@0"
I20260619 16:58:53.650980 29131 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:58:53.651908 29146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:53.652115 29131 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:53.652240 29131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-1/wal
uuid: "464343bddb8a42d39d18253f7ddfc32a"
format_stamp: "Formatted at 2026-06-19 16:58:53 on dist-test-slave-vnbj"
server_key: "f25def7f772e5748bdc9eb4e565243ef"
server_key_iv: "d2a42d126cd36dc92de854a728a3add1"
server_key_version: "encryptionkey@0"
I20260619 16:58:53.652321 29131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:53.666761 29131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:53.669203 29131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:53.669379 29131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:53.669972 29131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:53.670014 29131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:53.670038 29131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:53.670051 29131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 16:58:53 dist-test-slave-vnbj krb5kdc[28874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888333, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM
I20260619 16:58:53.679030 29131 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.130:45993
I20260619 16:58:53.679111 29261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.130:45993 every 8 connection(s)
I20260619 16:58:53.679428 29131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
I20260619 16:58:53.680133 28969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:53.670449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:40089 (local address 127.20.231.190:41933)
0619 16:58:53.670501 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:53.670503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:53.670728 (+ 225us) server_negotiation.cc:407] Connection header received
0619 16:58:53.671357 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:53.671359 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:53.671371 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:53.671427 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:53.672231 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.672779 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.673851 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.674084 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.674176 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:53.676886 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:53.676889 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:53.676889 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:53.676898 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:53.678460 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:53.678978 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:53.678979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:53.678979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:53.679059 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:53.679401 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:53.679402 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:53.679403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:53.679566 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:53.679624 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:53.679956 (+ 332us) server_negotiation.cc:299] Negotiation successful
0619 16:58:53.680042 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 16:58:53.680333 29155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:53.670448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41933 (local address 127.20.231.130:40089)
0619 16:58:53.670605 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:53.670643 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:53.671241 (+ 598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:53.671501 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:53.671506 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:53.671856 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:53.672128 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:53.672135 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.672878 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.672879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:53.673749 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:53.673752 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.674193 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.674194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:53.674338 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:53.674936 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:53.674948 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:53.676780 (+ 1832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:53.678559 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:53.678567 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:53.678569 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:53.678887 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:53.679167 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:53.679168 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:53.679169 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:53.679323 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:53.679626 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:53.679633 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:53.679817 (+ 184us) client_negotiation.cc:769] Sending connection context
0619 16:58:53.679922 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 16:58:53.680089 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":83,"threads_started":1}
I20260619 16:58:53.680807 29262 heartbeater.cc:347] Connected to a master server at 127.20.231.190:41933
I20260619 16:58:53.680955 29262 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:53.681192 29262 heartbeater.cc:511] Master 127.20.231.190:41933 requested a full tablet report, sending...
I20260619 16:58:53.681701 28919 ts_manager.cc:194] Registered new tserver with Master: 464343bddb8a42d39d18253f7ddfc32a (127.20.231.130:45993)
I20260619 16:58:53.682271 28919 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.231.130@KRBTEST.COM'} at 127.20.231.130:40089
I20260619 16:58:53.687899 28969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:53.682013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:60467 (local address 127.20.231.190:41933)
0619 16:58:53.682056 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:53.682058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:53.682113 (+ 55us) server_negotiation.cc:407] Connection header received
0619 16:58:53.682178 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:53.682179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:53.682187 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:53.682232 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:53.682611 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.683217 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.684027 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.684212 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.684285 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:53.685237 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:53.685238 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:53.685238 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:53.685245 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:53.686700 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:53.687142 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:53.687144 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:53.687144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:53.687209 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:53.687416 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:53.687417 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:53.687418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:53.687583 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:53.687645 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:53.687771 (+ 126us) server_negotiation.cc:299] Negotiation successful
0619 16:58:53.687833 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 16:58:53.687932 29268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:53.681920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41933 (local address 127.20.231.130:60467)
0619 16:58:53.682054 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:53.682066 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:53.682127 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:53.682237 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:53.682238 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:53.682457 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:53.682547 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:53.682550 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.683322 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.683323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:53.683932 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:53.683934 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.684293 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.684294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:53.684349 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:53.684844 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:53.684849 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:53.685144 (+ 295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:53.686808 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:53.686829 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:53.686829 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:53.687060 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:53.687275 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:53.687276 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:53.687276 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:53.687335 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:53.687646 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:53.687647 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:53.687692 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 16:58:53.687746 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 16:58:53.687852 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":78,"threads_started":1}
I20260619 16:58:53.689529 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 29131
I20260619 16:58:53.689754 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance
I20260619 16:58:53.689942 21406 external_mini_cluster.cc:1428] Setting key d877c5555d047d6297e3c1647c7869c5
WARNING: no policy specified for oryx/127.20.231.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.231.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.231.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.131.keytab.
Entry for principal oryx/127.20.231.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.131.keytab.
I20260619 16:58:53.718339 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.131:0
--local_ip_for_outbound_sockets=127.20.231.131
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:41933
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.131.keytab
--principal=oryx/127.20.231.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:35773
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:53.801467 29271 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:53.801700 29271 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:53.801728 29271 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:53.803025 29271 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:53.803076 29271 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:53.803098 29271 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 16:58:53.803151 29271 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.131
I20260619 16:58:53.804774 29271 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:35773
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.131.keytab
--principal=oryx/127.20.231.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.231.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.20.231.190:41933
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29271
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.131
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:53.805047 29271 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:53.805333 29271 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:53.810703 29279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:53.810707 29276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:53.810956 29271 server_base.cc:1034] running on GCE node
W20260619 16:58:53.811148 29277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:53.811444 29271 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:53.811723 29271 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:53.812911 29271 hybrid_clock.cc:648] HybridClock initialized: now 1781888333812903 us; error 37 us; skew 500 ppm
Jun 19 16:58:53 dist-test-slave-vnbj krb5kdc[28874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888333, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:53.815649 29271 init.cc:377] Logged in from keytab as oryx/127.20.231.131@KRBTEST.COM (short username oryx)
I20260619 16:58:53.816120 29271 webserver.cc:460] Webserver started at http://127.20.231.131:40231/ using document root <none> and password file <none>
I20260619 16:58:53.816370 29271 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:53.816462 29271 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:53.816599 29271 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:53.817692 29271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-2/data/instance:
uuid: "3acbb99594704efeab28df68782b198a"
format_stamp: "Formatted at 2026-06-19 16:58:53 on dist-test-slave-vnbj"
server_key: "078e450048b9b1459593d10257a47701"
server_key_iv: "9e051fd9ad03127b3cb5749932200199"
server_key_version: "encryptionkey@0"
I20260619 16:58:53.818058 29271 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance:
uuid: "3acbb99594704efeab28df68782b198a"
format_stamp: "Formatted at 2026-06-19 16:58:53 on dist-test-slave-vnbj"
server_key: "078e450048b9b1459593d10257a47701"
server_key_iv: "9e051fd9ad03127b3cb5749932200199"
server_key_version: "encryptionkey@0"
I20260619 16:58:53.819554 29271 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 16:58:53.820305 29286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:53.820473 29271 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:58:53.820564 29271 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-2/wal
uuid: "3acbb99594704efeab28df68782b198a"
format_stamp: "Formatted at 2026-06-19 16:58:53 on dist-test-slave-vnbj"
server_key: "078e450048b9b1459593d10257a47701"
server_key_iv: "9e051fd9ad03127b3cb5749932200199"
server_key_version: "encryptionkey@0"
I20260619 16:58:53.820639 29271 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:53.838691 29271 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:53.840799 29271 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:53.840966 29271 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:53.841586 29271 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:53.841642 29271 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:53.841678 29271 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:53.841722 29271 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 16:58:53 dist-test-slave-vnbj krb5kdc[28874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888333, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM
I20260619 16:58:53.848919 29271 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.131:46261
I20260619 16:58:53.849308 29271 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
I20260619 16:58:53.849305 29401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.131:46261 every 8 connection(s)
I20260619 16:58:53.851680 28969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:53.841970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:47747 (local address 127.20.231.190:41933)
0619 16:58:53.842042 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:53.842045 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:53.842309 (+ 264us) server_negotiation.cc:407] Connection header received
0619 16:58:53.842839 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:53.842840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:53.842869 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:53.842911 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:53.843878 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.844450 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.845527 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.845725 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.845802 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:53.848712 (+ 2910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:53.848714 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:53.848714 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:53.848723 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:53.850290 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:53.850696 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:53.850698 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:53.850698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:53.850764 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:53.851027 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:53.851028 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:53.851028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:53.851150 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:53.851223 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:53.851501 (+ 278us) server_negotiation.cc:299] Negotiation successful
0619 16:58:53.851583 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 16:58:53.851871 29295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:53.841972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41933 (local address 127.20.231.131:47747)
0619 16:58:53.842142 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:53.842184 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:53.842716 (+ 532us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:53.842991 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:53.842995 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:53.843378 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:53.843776 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:53.843782 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.844555 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.844556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:53.845415 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:53.845417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.845810 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.845811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:53.845945 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:53.846525 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:53.846533 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:53.848609 (+ 2076us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:53.850391 (+ 1782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:53.850396 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:53.850399 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:53.850621 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:53.850828 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:53.850829 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:53.850829 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:53.850960 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:53.851214 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:53.851220 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:53.851372 (+ 152us) client_negotiation.cc:769] Sending connection context
0619 16:58:53.851481 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 16:58:53.851648 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":89,"threads_started":1}
I20260619 16:58:53.852365 29402 heartbeater.cc:347] Connected to a master server at 127.20.231.190:41933
I20260619 16:58:53.852453 29402 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:53.852651 29402 heartbeater.cc:511] Master 127.20.231.190:41933 requested a full tablet report, sending...
I20260619 16:58:53.853020 28919 ts_manager.cc:194] Registered new tserver with Master: 3acbb99594704efeab28df68782b198a (127.20.231.131:46261)
I20260619 16:58:53.853411 28919 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.20.231.131@KRBTEST.COM'} at 127.20.231.131:47747
I20260619 16:58:53.853773 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 29271
I20260619 16:58:53.853868 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance
I20260619 16:58:53.854053 21406 external_mini_cluster.cc:1428] Setting key 2da46f2a62939b6fbfb9fb287d8e5d2b
I20260619 16:58:53.855485 21406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 16:58:53.859740 28969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:53.853527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:33087 (local address 127.20.231.190:41933)
0619 16:58:53.853570 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:53.853572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:53.853702 (+ 130us) server_negotiation.cc:407] Connection header received
0619 16:58:53.853785 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:53.853786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:53.853795 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:53.853850 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:53.854170 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.854691 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.855591 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.855794 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.855871 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:53.856825 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:53.856826 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:53.856826 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:53.856833 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:53.858442 (+ 1609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:53.858791 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:53.858792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:53.858792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:53.858883 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:53.859070 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:53.859071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:53.859071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:53.859226 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:53.859296 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:53.859545 (+ 249us) server_negotiation.cc:299] Negotiation successful
0619 16:58:53.859631 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 16:58:53.859781 29408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:53.853434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41933 (local address 127.20.231.131:33087)
0619 16:58:53.853636 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:53.853652 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:53.853717 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:53.853856 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:53.853856 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:53.854041 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:53.854097 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:53.854098 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.854795 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.854796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:53.855502 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:53.855504 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.855873 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.855874 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:53.855940 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:53.856407 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:53.856412 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:53.856719 (+ 307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:53.858527 (+ 1808us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:53.858527 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:53.858528 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:53.858711 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:53.858954 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:53.858954 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:53.858955 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:53.859010 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:53.859364 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:53.859366 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:53.859411 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 16:58:53.859494 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 16:58:53.859590 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":81,"threads_started":1}
Jun 19 16:58:53 dist-test-slave-vnbj krb5kdc[28874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888333, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 16:58:53 dist-test-slave-vnbj krb5kdc[28874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:58:53 dist-test-slave-vnbj krb5kdc[28874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
W20260619 16:58:53.879166 28969 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:53.873059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49948 (local address 127.20.231.190:41933)
0619 16:58:53.873122 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:53.873124 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:53.873139 (+ 15us) server_negotiation.cc:407] Connection header received
0619 16:58:53.873166 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:53.873166 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:53.873174 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:53.873238 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:53.873804 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.874318 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.875263 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.875453 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.875519 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:53.879050 (+ 3531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49948: BlockingRecv error: recv got EOF from 127.0.0.1:49948 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Jun 19 16:58:53 dist-test-slave-vnbj krb5kdc[28874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888333, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM
I20260619 16:58:53.888316 28969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:53.880082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49950 (local address 127.20.231.190:41933)
0619 16:58:53.880158 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:53.880161 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:53.880211 (+ 50us) server_negotiation.cc:407] Connection header received
0619 16:58:53.880267 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:53.880267 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:53.880304 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:53.880362 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:53.881011 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.881508 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.882299 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.882498 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.882563 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:53.885142 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:53.885144 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:53.885144 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:53.885151 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:53.886740 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:53.887132 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:53.887134 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:53.887134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:53.887210 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:53.887757 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:53.887758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:53.887759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:53.887896 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:53.887992 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:53.888158 (+ 166us) server_negotiation.cc:299] Negotiation successful
0619 16:58:53.888238 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 16:58:53.890136 28919 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49950:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:58:53.891745 28919 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:58:53.898375 29425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:53.894120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:46675 (local address 127.0.0.1:46354)
0619 16:58:53.894279 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:53.894321 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:53.894384 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:53.894746 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:53.894749 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:53.894752 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:53.894883 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:53.894886 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.896285 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.896286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:53.897356 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:53.897358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.898044 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.898045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:53.898137 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:53.898151 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 16:58:53.898226 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 16:58:53.898282 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"mutex_wait_us":27,"thread_start_us":45,"threads_started":1}
I20260619 16:58:53.898384 29426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:53.894261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46354 (local address 127.20.231.129:46675)
0619 16:58:53.894552 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:53.894558 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:53.894569 (+ 11us) server_negotiation.cc:407] Connection header received
0619 16:58:53.894612 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:53.894614 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:53.894632 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:53.894967 (+ 335us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:53.895029 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.896162 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.897442 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.897948 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.898067 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:53.898130 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:53.898241 (+ 111us) server_negotiation.cc:299] Negotiation successful
0619 16:58:53.898286 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":80,"threads_started":1}
I20260619 16:58:53.898501 29424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:53.894136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:45993 (local address 127.0.0.1:57890)
0619 16:58:53.894621 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:53.894632 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:53.894713 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:53.894996 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:53.894997 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:53.894999 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:53.895051 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:53.895053 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.895956 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.895956 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:53.897208 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:53.897210 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.898246 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.898248 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:53.898310 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:53.898315 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 16:58:53.898374 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 16:58:53.898416 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":87,"threads_started":1}
I20260619 16:58:53.898674 29429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:53.894442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53222 (local address 127.20.231.131:46261)
0619 16:58:53.894713 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:53.894717 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:53.894922 (+ 205us) server_negotiation.cc:407] Connection header received
0619 16:58:53.895043 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:53.895045 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:53.895061 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:53.895129 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:53.895306 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.896374 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.897500 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.898197 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.898282 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:53.898333 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:53.898539 (+ 206us) server_negotiation.cc:299] Negotiation successful
0619 16:58:53.898585 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":94,"threads_started":1}
I20260619 16:58:53.898789 29428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:53.894449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57890 (local address 127.20.231.130:45993)
0619 16:58:53.894619 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:53.894623 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:53.894704 (+ 81us) server_negotiation.cc:407] Connection header received
0619 16:58:53.894801 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:53.894804 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:53.894822 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:53.894927 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:53.895137 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.895878 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.897295 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.898062 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.898154 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:53.898209 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:53.898612 (+ 403us) server_negotiation.cc:299] Negotiation successful
0619 16:58:53.898657 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":88,"threads_started":1}
I20260619 16:58:53.898871 29427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:53.894357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:46261 (local address 127.0.0.1:53222)
0619 16:58:53.894812 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:53.894821 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:53.894949 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:53.895194 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:53.895194 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:53.895196 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:53.895226 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:53.895227 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.896455 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.896456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:53.897409 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:53.897411 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.898366 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.898367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:53.898417 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:53.898419 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 16:58:53.898464 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 16:58:53.898494 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"mutex_wait_us":1,"thread_start_us":77,"threads_started":1}
I20260619 16:58:53.899531 29056 tablet_service.cc:1467] Processing CreateTablet for tablet 44c31fd3868846048678b369f4010812 (DEFAULT_TABLE table=test-table [id=ead0f60f3cf94a22a8401edb22af9510]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:53.899513 29336 tablet_service.cc:1467] Processing CreateTablet for tablet 44c31fd3868846048678b369f4010812 (DEFAULT_TABLE table=test-table [id=ead0f60f3cf94a22a8401edb22af9510]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:53.899513 29183 tablet_service.cc:1467] Processing CreateTablet for tablet 44c31fd3868846048678b369f4010812 (DEFAULT_TABLE table=test-table [id=ead0f60f3cf94a22a8401edb22af9510]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:53.899909 29056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44c31fd3868846048678b369f4010812. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:53.899911 29336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44c31fd3868846048678b369f4010812. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:53.899911 29183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44c31fd3868846048678b369f4010812. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:53.903195 29430 tablet_bootstrap.cc:492] T 44c31fd3868846048678b369f4010812 P 5c9cb6210ca6432088b94e62e5b395ac: Bootstrap starting.
I20260619 16:58:53.903601 29431 tablet_bootstrap.cc:492] T 44c31fd3868846048678b369f4010812 P 3acbb99594704efeab28df68782b198a: Bootstrap starting.
I20260619 16:58:53.903972 29432 tablet_bootstrap.cc:492] T 44c31fd3868846048678b369f4010812 P 464343bddb8a42d39d18253f7ddfc32a: Bootstrap starting.
I20260619 16:58:53.904155 29430 tablet_bootstrap.cc:654] T 44c31fd3868846048678b369f4010812 P 5c9cb6210ca6432088b94e62e5b395ac: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:53.904218 29431 tablet_bootstrap.cc:654] T 44c31fd3868846048678b369f4010812 P 3acbb99594704efeab28df68782b198a: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:53.904533 29430 log.cc:824] T 44c31fd3868846048678b369f4010812 P 5c9cb6210ca6432088b94e62e5b395ac: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:53.904533 29431 log.cc:824] T 44c31fd3868846048678b369f4010812 P 3acbb99594704efeab28df68782b198a: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:53.904557 29432 tablet_bootstrap.cc:654] T 44c31fd3868846048678b369f4010812 P 464343bddb8a42d39d18253f7ddfc32a: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:53.904889 29432 log.cc:824] T 44c31fd3868846048678b369f4010812 P 464343bddb8a42d39d18253f7ddfc32a: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:53.905306 29431 tablet_bootstrap.cc:492] T 44c31fd3868846048678b369f4010812 P 3acbb99594704efeab28df68782b198a: No bootstrap required, opened a new log
I20260619 16:58:53.905329 29430 tablet_bootstrap.cc:492] T 44c31fd3868846048678b369f4010812 P 5c9cb6210ca6432088b94e62e5b395ac: No bootstrap required, opened a new log
I20260619 16:58:53.905381 29430 ts_tablet_manager.cc:1397] T 44c31fd3868846048678b369f4010812 P 5c9cb6210ca6432088b94e62e5b395ac: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 16:58:53.905381 29431 ts_tablet_manager.cc:1397] T 44c31fd3868846048678b369f4010812 P 3acbb99594704efeab28df68782b198a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:58:53.905719 29432 tablet_bootstrap.cc:492] T 44c31fd3868846048678b369f4010812 P 464343bddb8a42d39d18253f7ddfc32a: No bootstrap required, opened a new log
I20260619 16:58:53.905853 29432 ts_tablet_manager.cc:1397] T 44c31fd3868846048678b369f4010812 P 464343bddb8a42d39d18253f7ddfc32a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 16:58:53.907409 29432 raft_consensus.cc:348] T 44c31fd3868846048678b369f4010812 P 464343bddb8a42d39d18253f7ddfc32a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } }
I20260619 16:58:53.907560 29432 raft_consensus.cc:374] T 44c31fd3868846048678b369f4010812 P 464343bddb8a42d39d18253f7ddfc32a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:53.907603 29432 raft_consensus.cc:729] T 44c31fd3868846048678b369f4010812 P 464343bddb8a42d39d18253f7ddfc32a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 464343bddb8a42d39d18253f7ddfc32a, State: Initialized, Role: FOLLOWER
I20260619 16:58:53.907526 29431 raft_consensus.cc:348] T 44c31fd3868846048678b369f4010812 P 3acbb99594704efeab28df68782b198a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } }
I20260619 16:58:53.907527 29430 raft_consensus.cc:348] T 44c31fd3868846048678b369f4010812 P 5c9cb6210ca6432088b94e62e5b395ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } }
I20260619 16:58:53.907685 29430 raft_consensus.cc:374] T 44c31fd3868846048678b369f4010812 P 5c9cb6210ca6432088b94e62e5b395ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:53.907694 29431 raft_consensus.cc:374] T 44c31fd3868846048678b369f4010812 P 3acbb99594704efeab28df68782b198a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:53.907732 29431 raft_consensus.cc:729] T 44c31fd3868846048678b369f4010812 P 3acbb99594704efeab28df68782b198a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3acbb99594704efeab28df68782b198a, State: Initialized, Role: FOLLOWER
I20260619 16:58:53.907732 29430 raft_consensus.cc:729] T 44c31fd3868846048678b369f4010812 P 5c9cb6210ca6432088b94e62e5b395ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c9cb6210ca6432088b94e62e5b395ac, State: Initialized, Role: FOLLOWER
I20260619 16:58:53.907737 29432 consensus_queue.cc:260] T 44c31fd3868846048678b369f4010812 P 464343bddb8a42d39d18253f7ddfc32a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } }
I20260619 16:58:53.907846 29430 consensus_queue.cc:260] T 44c31fd3868846048678b369f4010812 P 5c9cb6210ca6432088b94e62e5b395ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } }
I20260619 16:58:53.907861 29431 consensus_queue.cc:260] T 44c31fd3868846048678b369f4010812 P 3acbb99594704efeab28df68782b198a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } }
I20260619 16:58:53.908008 29432 ts_tablet_manager.cc:1428] T 44c31fd3868846048678b369f4010812 P 464343bddb8a42d39d18253f7ddfc32a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 16:58:53.908100 29430 ts_tablet_manager.cc:1428] T 44c31fd3868846048678b369f4010812 P 5c9cb6210ca6432088b94e62e5b395ac: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 16:58:53.908100 29431 ts_tablet_manager.cc:1428] T 44c31fd3868846048678b369f4010812 P 3acbb99594704efeab28df68782b198a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 16:58:53.908140 29262 heartbeater.cc:503] Master 127.20.231.190:41933 was elected leader, sending a full tablet report...
I20260619 16:58:53.908231 29402 heartbeater.cc:503] Master 127.20.231.190:41933 was elected leader, sending a full tablet report...
I20260619 16:58:53.908336 29122 heartbeater.cc:503] Master 127.20.231.190:41933 was elected leader, sending a full tablet report...
I20260619 16:58:53.914134 29437 raft_consensus.cc:482] T 44c31fd3868846048678b369f4010812 P 5c9cb6210ca6432088b94e62e5b395ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:58:53.914250 29437 raft_consensus.cc:504] T 44c31fd3868846048678b369f4010812 P 5c9cb6210ca6432088b94e62e5b395ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } }
I20260619 16:58:53.914592 29437 leader_election.cc:288] T 44c31fd3868846048678b369f4010812 P 5c9cb6210ca6432088b94e62e5b395ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3acbb99594704efeab28df68782b198a (127.20.231.131:46261), 464343bddb8a42d39d18253f7ddfc32a (127.20.231.130:45993)
I20260619 16:58:53.917588 29015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:53.914799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:46261 (local address 127.20.231.129:55065)
0619 16:58:53.914911 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:53.914923 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:53.914969 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:53.915117 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:53.915118 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:53.915120 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:53.915199 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:53.915201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.915918 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.915919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:53.916766 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:53.916768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.917325 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.917326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:53.917417 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:53.917422 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 16:58:53.917471 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 16:58:53.917502 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260619 16:58:53.917610 29429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:53.914871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:55065 (local address 127.20.231.131:46261)
0619 16:58:53.914985 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:53.914988 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:53.914995 (+ 7us) server_negotiation.cc:407] Connection header received
0619 16:58:53.915021 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:53.915022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:53.915032 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:53.915105 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:53.915284 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.915841 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.916852 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.917255 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.917309 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:53.917316 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:53.917496 (+ 180us) server_negotiation.cc:299] Negotiation successful
0619 16:58:53.917533 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260619 16:58:53.917703 29428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:53.914930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:47181 (local address 127.20.231.130:45993)
0619 16:58:53.914981 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:53.914983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:53.915118 (+ 135us) server_negotiation.cc:407] Connection header received
0619 16:58:53.915172 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:53.915172 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:53.915181 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:53.915233 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:53.915462 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.915989 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.916959 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.917343 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.917410 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:53.917417 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:53.917596 (+ 179us) server_negotiation.cc:299] Negotiation successful
0619 16:58:53.917643 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 16:58:53.917790 29439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:53.914833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:45993 (local address 127.20.231.129:47181)
0619 16:58:53.915059 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:53.915066 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:53.915123 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:53.915241 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:53.915242 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:53.915244 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:53.915399 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:53.915401 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.916069 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.916070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:53.916879 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:53.916882 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.917428 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.917429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:53.917523 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:53.917527 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:58:53.917579 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 16:58:53.917615 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":120,"threads_started":1}
I20260619 16:58:53.918000 29356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44c31fd3868846048678b369f4010812" candidate_uuid: "5c9cb6210ca6432088b94e62e5b395ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3acbb99594704efeab28df68782b198a" is_pre_election: true
I20260619 16:58:53.918056 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44c31fd3868846048678b369f4010812" candidate_uuid: "5c9cb6210ca6432088b94e62e5b395ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "464343bddb8a42d39d18253f7ddfc32a" is_pre_election: true
I20260619 16:58:53.918186 29216 raft_consensus.cc:2454] T 44c31fd3868846048678b369f4010812 P 464343bddb8a42d39d18253f7ddfc32a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c9cb6210ca6432088b94e62e5b395ac in term 0.
I20260619 16:58:53.918181 29356 raft_consensus.cc:2454] T 44c31fd3868846048678b369f4010812 P 3acbb99594704efeab28df68782b198a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c9cb6210ca6432088b94e62e5b395ac in term 0.
I20260619 16:58:53.918404 29008 leader_election.cc:302] T 44c31fd3868846048678b369f4010812 P 5c9cb6210ca6432088b94e62e5b395ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3acbb99594704efeab28df68782b198a, 5c9cb6210ca6432088b94e62e5b395ac; no voters:
I20260619 16:58:53.918560 29437 raft_consensus.cc:2781] T 44c31fd3868846048678b369f4010812 P 5c9cb6210ca6432088b94e62e5b395ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:58:53.918620 29437 raft_consensus.cc:482] T 44c31fd3868846048678b369f4010812 P 5c9cb6210ca6432088b94e62e5b395ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:58:53.918665 29437 raft_consensus.cc:3037] T 44c31fd3868846048678b369f4010812 P 5c9cb6210ca6432088b94e62e5b395ac [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:53.919646 29437 raft_consensus.cc:504] T 44c31fd3868846048678b369f4010812 P 5c9cb6210ca6432088b94e62e5b395ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } }
I20260619 16:58:53.919797 29437 leader_election.cc:288] T 44c31fd3868846048678b369f4010812 P 5c9cb6210ca6432088b94e62e5b395ac [CANDIDATE]: Term 1 election: Requested vote from peers 3acbb99594704efeab28df68782b198a (127.20.231.131:46261), 464343bddb8a42d39d18253f7ddfc32a (127.20.231.130:45993)
I20260619 16:58:53.919926 29356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44c31fd3868846048678b369f4010812" candidate_uuid: "5c9cb6210ca6432088b94e62e5b395ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3acbb99594704efeab28df68782b198a"
I20260619 16:58:53.919958 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44c31fd3868846048678b369f4010812" candidate_uuid: "5c9cb6210ca6432088b94e62e5b395ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "464343bddb8a42d39d18253f7ddfc32a"
I20260619 16:58:53.920006 29356 raft_consensus.cc:3037] T 44c31fd3868846048678b369f4010812 P 3acbb99594704efeab28df68782b198a [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:53.920068 29216 raft_consensus.cc:3037] T 44c31fd3868846048678b369f4010812 P 464343bddb8a42d39d18253f7ddfc32a [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:53.920692 29356 raft_consensus.cc:2454] T 44c31fd3868846048678b369f4010812 P 3acbb99594704efeab28df68782b198a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c9cb6210ca6432088b94e62e5b395ac in term 1.
I20260619 16:58:53.920847 29216 raft_consensus.cc:2454] T 44c31fd3868846048678b369f4010812 P 464343bddb8a42d39d18253f7ddfc32a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c9cb6210ca6432088b94e62e5b395ac in term 1.
I20260619 16:58:53.920855 29008 leader_election.cc:302] T 44c31fd3868846048678b369f4010812 P 5c9cb6210ca6432088b94e62e5b395ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3acbb99594704efeab28df68782b198a, 5c9cb6210ca6432088b94e62e5b395ac; no voters:
I20260619 16:58:53.920953 29437 raft_consensus.cc:2781] T 44c31fd3868846048678b369f4010812 P 5c9cb6210ca6432088b94e62e5b395ac [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:53.921087 29437 raft_consensus.cc:686] T 44c31fd3868846048678b369f4010812 P 5c9cb6210ca6432088b94e62e5b395ac [term 1 LEADER]: Becoming Leader. State: Replica: 5c9cb6210ca6432088b94e62e5b395ac, State: Running, Role: LEADER
I20260619 16:58:53.921185 29437 consensus_queue.cc:237] T 44c31fd3868846048678b369f4010812 P 5c9cb6210ca6432088b94e62e5b395ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } }
I20260619 16:58:53.922032 28917 catalog_manager.cc:5495] T 44c31fd3868846048678b369f4010812 P 5c9cb6210ca6432088b94e62e5b395ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c9cb6210ca6432088b94e62e5b395ac (127.20.231.129). New cstate: current_term: 1 leader_uuid: "5c9cb6210ca6432088b94e62e5b395ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } health_report { overall_health: UNKNOWN } } }
W20260619 16:58:53.930060 29263 tablet.cc:2369] T 44c31fd3868846048678b369f4010812 P 464343bddb8a42d39d18253f7ddfc32a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:58:53.934089 29426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:53.931939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46360 (local address 127.20.231.129:46675)
0619 16:58:53.931996 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:53.931998 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:53.932028 (+ 30us) server_negotiation.cc:407] Connection header received
0619 16:58:53.932059 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:53.932060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:53.932075 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:53.932130 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:58:53.932293 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.932746 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.933466 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:53.933647 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:53.933697 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:53.933831 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:58:53.933924 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:58:53.933983 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:53.934017 (+ 34us) server_negotiation.cc:299] Negotiation successful
0619 16:58:53.934033 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 16:58:53.935995 29356 raft_consensus.cc:1261] T 44c31fd3868846048678b369f4010812 P 3acbb99594704efeab28df68782b198a [term 1 FOLLOWER]: Refusing update from remote peer 5c9cb6210ca6432088b94e62e5b395ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:53.935995 29216 raft_consensus.cc:1261] T 44c31fd3868846048678b369f4010812 P 464343bddb8a42d39d18253f7ddfc32a [term 1 FOLLOWER]: Refusing update from remote peer 5c9cb6210ca6432088b94e62e5b395ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:53.936367 29440 consensus_queue.cc:1035] T 44c31fd3868846048678b369f4010812 P 5c9cb6210ca6432088b94e62e5b395ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:53.936494 29437 consensus_queue.cc:1035] T 44c31fd3868846048678b369f4010812 P 5c9cb6210ca6432088b94e62e5b395ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:53.939169 29443 mvcc.cc:205] Tried to move back new op lower bound from 7298614615798755328 to 7298614615741706240. Current Snapshot: MvccSnapshot[applied={T|T < 7298614615798755328}]
I20260619 16:58:53.939419 29444 mvcc.cc:205] Tried to move back new op lower bound from 7298614615798755328 to 7298614615741706240. Current Snapshot: MvccSnapshot[applied={T|T < 7298614615798755328}]
I20260619 16:58:53.940116 29446 mvcc.cc:205] Tried to move back new op lower bound from 7298614615798755328 to 7298614615741706240. Current Snapshot: MvccSnapshot[applied={T|T < 7298614615798755328}]
I20260619 16:58:53.941622 28917 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:58:53.941723 28917 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:58:53.942580 28917 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4616d2b9af7648f9ac7f8bcc6a5e185d: Sending DeleteTablet for 3 replicas of tablet 44c31fd3868846048678b369f4010812
I20260619 16:58:53.942929 29183 tablet_service.cc:1514] Processing DeleteTablet for tablet 44c31fd3868846048678b369f4010812 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:53 UTC) from {username='oryx', principal='oryx/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:57890
I20260619 16:58:53.942992 29056 tablet_service.cc:1514] Processing DeleteTablet for tablet 44c31fd3868846048678b369f4010812 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:53 UTC) from {username='oryx', principal='oryx/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:46354
I20260619 16:58:53.943274 29336 tablet_service.cc:1514] Processing DeleteTablet for tablet 44c31fd3868846048678b369f4010812 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:53 UTC) from {username='oryx', principal='oryx/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:53222
I20260619 16:58:53.943466 29452 tablet_replica.cc:330] T 44c31fd3868846048678b369f4010812 P 464343bddb8a42d39d18253f7ddfc32a: stopping tablet replica
I20260619 16:58:53.943498 29454 tablet_replica.cc:330] T 44c31fd3868846048678b369f4010812 P 3acbb99594704efeab28df68782b198a: stopping tablet replica
I20260619 16:58:53.943501 28917 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49950:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 16:58:53.943589 29452 raft_consensus.cc:2229] T 44c31fd3868846048678b369f4010812 P 464343bddb8a42d39d18253f7ddfc32a [term 1 FOLLOWER]: Raft consensus shutting down.
W20260619 16:58:53.943660 28917 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:58:53.943709 29452 raft_consensus.cc:2258] T 44c31fd3868846048678b369f4010812 P 464343bddb8a42d39d18253f7ddfc32a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:53.943985 29454 raft_consensus.cc:2229] T 44c31fd3868846048678b369f4010812 P 3acbb99594704efeab28df68782b198a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:58:53.944232 29452 ts_tablet_manager.cc:1905] T 44c31fd3868846048678b369f4010812 P 464343bddb8a42d39d18253f7ddfc32a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:53.944308 29453 tablet_replica.cc:330] T 44c31fd3868846048678b369f4010812 P 5c9cb6210ca6432088b94e62e5b395ac: stopping tablet replica
I20260619 16:58:53.944394 29453 raft_consensus.cc:2229] T 44c31fd3868846048678b369f4010812 P 5c9cb6210ca6432088b94e62e5b395ac [term 1 LEADER]: Raft consensus shutting down.
I20260619 16:58:53.944509 29453 raft_consensus.cc:2258] T 44c31fd3868846048678b369f4010812 P 5c9cb6210ca6432088b94e62e5b395ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:53.944172 29454 raft_consensus.cc:2258] T 44c31fd3868846048678b369f4010812 P 3acbb99594704efeab28df68782b198a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:53.945308 29183 tablet_service.cc:1467] Processing CreateTablet for tablet 92b707f03b204711af02d57bdd4fa62b (DEFAULT_TABLE table=test-table [id=95de3dc5aee54b17b82c23dc493fdef1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:53.945362 29056 tablet_service.cc:1467] Processing CreateTablet for tablet 92b707f03b204711af02d57bdd4fa62b (DEFAULT_TABLE table=test-table [id=95de3dc5aee54b17b82c23dc493fdef1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:53.945429 29183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92b707f03b204711af02d57bdd4fa62b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:53.945494 29056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92b707f03b204711af02d57bdd4fa62b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:53.945725 29336 tablet_service.cc:1467] Processing CreateTablet for tablet 92b707f03b204711af02d57bdd4fa62b (DEFAULT_TABLE table=test-table [id=95de3dc5aee54b17b82c23dc493fdef1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:53.945822 29336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92b707f03b204711af02d57bdd4fa62b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:53.946314 29452 ts_tablet_manager.cc:1918] T 44c31fd3868846048678b369f4010812 P 464343bddb8a42d39d18253f7ddfc32a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:58:53.946388 29452 log.cc:1196] T 44c31fd3868846048678b369f4010812 P 464343bddb8a42d39d18253f7ddfc32a: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-1/wal/wals/44c31fd3868846048678b369f4010812
I20260619 16:58:53.946674 29452 ts_tablet_manager.cc:1939] T 44c31fd3868846048678b369f4010812 P 464343bddb8a42d39d18253f7ddfc32a: Deleting consensus metadata
I20260619 16:58:53.946970 29432 tablet_bootstrap.cc:492] T 92b707f03b204711af02d57bdd4fa62b P 464343bddb8a42d39d18253f7ddfc32a: Bootstrap starting.
I20260619 16:58:53.947062 29430 tablet_bootstrap.cc:492] T 92b707f03b204711af02d57bdd4fa62b P 5c9cb6210ca6432088b94e62e5b395ac: Bootstrap starting.
I20260619 16:58:53.947373 29432 tablet_bootstrap.cc:654] T 92b707f03b204711af02d57bdd4fa62b P 464343bddb8a42d39d18253f7ddfc32a: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:53.947530 29430 tablet_bootstrap.cc:654] T 92b707f03b204711af02d57bdd4fa62b P 5c9cb6210ca6432088b94e62e5b395ac: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:53.948009 29432 tablet_bootstrap.cc:492] T 92b707f03b204711af02d57bdd4fa62b P 464343bddb8a42d39d18253f7ddfc32a: No bootstrap required, opened a new log
I20260619 16:58:53.948060 29432 ts_tablet_manager.cc:1397] T 92b707f03b204711af02d57bdd4fa62b P 464343bddb8a42d39d18253f7ddfc32a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:53.948076 28906 catalog_manager.cc:4841] TS 464343bddb8a42d39d18253f7ddfc32a (127.20.231.130:45993): tablet 44c31fd3868846048678b369f4010812 (table test-table [id=ead0f60f3cf94a22a8401edb22af9510]) successfully deleted
I20260619 16:58:53.948148 29431 tablet_bootstrap.cc:492] T 92b707f03b204711af02d57bdd4fa62b P 3acbb99594704efeab28df68782b198a: Bootstrap starting.
I20260619 16:58:53.948212 29432 raft_consensus.cc:348] T 92b707f03b204711af02d57bdd4fa62b P 464343bddb8a42d39d18253f7ddfc32a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } }
I20260619 16:58:53.948284 29432 raft_consensus.cc:374] T 92b707f03b204711af02d57bdd4fa62b P 464343bddb8a42d39d18253f7ddfc32a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:53.948305 29432 raft_consensus.cc:729] T 92b707f03b204711af02d57bdd4fa62b P 464343bddb8a42d39d18253f7ddfc32a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 464343bddb8a42d39d18253f7ddfc32a, State: Initialized, Role: FOLLOWER
I20260619 16:58:53.948346 29432 consensus_queue.cc:260] T 92b707f03b204711af02d57bdd4fa62b P 464343bddb8a42d39d18253f7ddfc32a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } }
I20260619 16:58:53.948421 29432 ts_tablet_manager.cc:1428] T 92b707f03b204711af02d57bdd4fa62b P 464343bddb8a42d39d18253f7ddfc32a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:53.948436 29453 ts_tablet_manager.cc:1905] T 44c31fd3868846048678b369f4010812 P 5c9cb6210ca6432088b94e62e5b395ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:53.948549 29430 tablet_bootstrap.cc:492] T 92b707f03b204711af02d57bdd4fa62b P 5c9cb6210ca6432088b94e62e5b395ac: No bootstrap required, opened a new log
I20260619 16:58:53.948606 29430 ts_tablet_manager.cc:1397] T 92b707f03b204711af02d57bdd4fa62b P 5c9cb6210ca6432088b94e62e5b395ac: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:58:53.948621 29431 tablet_bootstrap.cc:654] T 92b707f03b204711af02d57bdd4fa62b P 3acbb99594704efeab28df68782b198a: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:53.948778 29430 raft_consensus.cc:348] T 92b707f03b204711af02d57bdd4fa62b P 5c9cb6210ca6432088b94e62e5b395ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } }
I20260619 16:58:53.948860 29430 raft_consensus.cc:374] T 92b707f03b204711af02d57bdd4fa62b P 5c9cb6210ca6432088b94e62e5b395ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:53.948885 29430 raft_consensus.cc:729] T 92b707f03b204711af02d57bdd4fa62b P 5c9cb6210ca6432088b94e62e5b395ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c9cb6210ca6432088b94e62e5b395ac, State: Initialized, Role: FOLLOWER
I20260619 16:58:53.948940 29430 consensus_queue.cc:260] T 92b707f03b204711af02d57bdd4fa62b P 5c9cb6210ca6432088b94e62e5b395ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } }
I20260619 16:58:53.949031 29430 ts_tablet_manager.cc:1428] T 92b707f03b204711af02d57bdd4fa62b P 5c9cb6210ca6432088b94e62e5b395ac: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 16:58:53.949656 29454 ts_tablet_manager.cc:1905] T 44c31fd3868846048678b369f4010812 P 3acbb99594704efeab28df68782b198a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:53.951344 29454 ts_tablet_manager.cc:1918] T 44c31fd3868846048678b369f4010812 P 3acbb99594704efeab28df68782b198a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:58:53.951426 29454 log.cc:1196] T 44c31fd3868846048678b369f4010812 P 3acbb99594704efeab28df68782b198a: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-2/wal/wals/44c31fd3868846048678b369f4010812
I20260619 16:58:53.951715 29454 ts_tablet_manager.cc:1939] T 44c31fd3868846048678b369f4010812 P 3acbb99594704efeab28df68782b198a: Deleting consensus metadata
I20260619 16:58:53.951952 29453 ts_tablet_manager.cc:1918] T 44c31fd3868846048678b369f4010812 P 5c9cb6210ca6432088b94e62e5b395ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:58:53.952080 29453 log.cc:1196] T 44c31fd3868846048678b369f4010812 P 5c9cb6210ca6432088b94e62e5b395ac: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-0/wal/wals/44c31fd3868846048678b369f4010812
I20260619 16:58:53.952199 28904 catalog_manager.cc:4841] TS 3acbb99594704efeab28df68782b198a (127.20.231.131:46261): tablet 44c31fd3868846048678b369f4010812 (table test-table [id=ead0f60f3cf94a22a8401edb22af9510]) successfully deleted
I20260619 16:58:53.952366 29453 ts_tablet_manager.cc:1939] T 44c31fd3868846048678b369f4010812 P 5c9cb6210ca6432088b94e62e5b395ac: Deleting consensus metadata
I20260619 16:58:53.952764 28904 catalog_manager.cc:4841] TS 5c9cb6210ca6432088b94e62e5b395ac (127.20.231.129:46675): tablet 44c31fd3868846048678b369f4010812 (table test-table [id=ead0f60f3cf94a22a8401edb22af9510]) successfully deleted
I20260619 16:58:53.955236 29431 tablet_bootstrap.cc:492] T 92b707f03b204711af02d57bdd4fa62b P 3acbb99594704efeab28df68782b198a: No bootstrap required, opened a new log
I20260619 16:58:53.955294 29431 ts_tablet_manager.cc:1397] T 92b707f03b204711af02d57bdd4fa62b P 3acbb99594704efeab28df68782b198a: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260619 16:58:53.955474 29431 raft_consensus.cc:348] T 92b707f03b204711af02d57bdd4fa62b P 3acbb99594704efeab28df68782b198a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } }
I20260619 16:58:53.955538 29431 raft_consensus.cc:374] T 92b707f03b204711af02d57bdd4fa62b P 3acbb99594704efeab28df68782b198a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:53.955552 29431 raft_consensus.cc:729] T 92b707f03b204711af02d57bdd4fa62b P 3acbb99594704efeab28df68782b198a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3acbb99594704efeab28df68782b198a, State: Initialized, Role: FOLLOWER
I20260619 16:58:53.955603 29431 consensus_queue.cc:260] T 92b707f03b204711af02d57bdd4fa62b P 3acbb99594704efeab28df68782b198a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } }
I20260619 16:58:53.955716 29431 ts_tablet_manager.cc:1428] T 92b707f03b204711af02d57bdd4fa62b P 3acbb99594704efeab28df68782b198a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
W20260619 16:58:54.002360 29123 tablet.cc:2369] T 92b707f03b204711af02d57bdd4fa62b P 5c9cb6210ca6432088b94e62e5b395ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:58:54.099509 29440 raft_consensus.cc:482] T 92b707f03b204711af02d57bdd4fa62b P 5c9cb6210ca6432088b94e62e5b395ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:58:54.099722 29440 raft_consensus.cc:504] T 92b707f03b204711af02d57bdd4fa62b P 5c9cb6210ca6432088b94e62e5b395ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } }
I20260619 16:58:54.099975 29440 leader_election.cc:288] T 92b707f03b204711af02d57bdd4fa62b P 5c9cb6210ca6432088b94e62e5b395ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 464343bddb8a42d39d18253f7ddfc32a (127.20.231.130:45993), 3acbb99594704efeab28df68782b198a (127.20.231.131:46261)
W20260619 16:58:54.100113 29403 tablet.cc:2369] T 92b707f03b204711af02d57bdd4fa62b P 3acbb99594704efeab28df68782b198a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:58:54.100208 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92b707f03b204711af02d57bdd4fa62b" candidate_uuid: "5c9cb6210ca6432088b94e62e5b395ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "464343bddb8a42d39d18253f7ddfc32a" is_pre_election: true
I20260619 16:58:54.100199 29356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92b707f03b204711af02d57bdd4fa62b" candidate_uuid: "5c9cb6210ca6432088b94e62e5b395ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3acbb99594704efeab28df68782b198a" is_pre_election: true
I20260619 16:58:54.100303 29216 raft_consensus.cc:2454] T 92b707f03b204711af02d57bdd4fa62b P 464343bddb8a42d39d18253f7ddfc32a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c9cb6210ca6432088b94e62e5b395ac in term 0.
I20260619 16:58:54.100335 29356 raft_consensus.cc:2454] T 92b707f03b204711af02d57bdd4fa62b P 3acbb99594704efeab28df68782b198a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c9cb6210ca6432088b94e62e5b395ac in term 0.
I20260619 16:58:54.100500 29010 leader_election.cc:302] T 92b707f03b204711af02d57bdd4fa62b P 5c9cb6210ca6432088b94e62e5b395ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 464343bddb8a42d39d18253f7ddfc32a, 5c9cb6210ca6432088b94e62e5b395ac; no voters:
I20260619 16:58:54.100641 29440 raft_consensus.cc:2781] T 92b707f03b204711af02d57bdd4fa62b P 5c9cb6210ca6432088b94e62e5b395ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:58:54.100704 29440 raft_consensus.cc:482] T 92b707f03b204711af02d57bdd4fa62b P 5c9cb6210ca6432088b94e62e5b395ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:58:54.100733 29440 raft_consensus.cc:3037] T 92b707f03b204711af02d57bdd4fa62b P 5c9cb6210ca6432088b94e62e5b395ac [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:54.101595 29440 raft_consensus.cc:504] T 92b707f03b204711af02d57bdd4fa62b P 5c9cb6210ca6432088b94e62e5b395ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } }
I20260619 16:58:54.101738 29440 leader_election.cc:288] T 92b707f03b204711af02d57bdd4fa62b P 5c9cb6210ca6432088b94e62e5b395ac [CANDIDATE]: Term 1 election: Requested vote from peers 464343bddb8a42d39d18253f7ddfc32a (127.20.231.130:45993), 3acbb99594704efeab28df68782b198a (127.20.231.131:46261)
I20260619 16:58:54.101975 29356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92b707f03b204711af02d57bdd4fa62b" candidate_uuid: "5c9cb6210ca6432088b94e62e5b395ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3acbb99594704efeab28df68782b198a"
I20260619 16:58:54.101975 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92b707f03b204711af02d57bdd4fa62b" candidate_uuid: "5c9cb6210ca6432088b94e62e5b395ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "464343bddb8a42d39d18253f7ddfc32a"
I20260619 16:58:54.102058 29216 raft_consensus.cc:3037] T 92b707f03b204711af02d57bdd4fa62b P 464343bddb8a42d39d18253f7ddfc32a [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:54.102058 29356 raft_consensus.cc:3037] T 92b707f03b204711af02d57bdd4fa62b P 3acbb99594704efeab28df68782b198a [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:54.102811 29216 raft_consensus.cc:2454] T 92b707f03b204711af02d57bdd4fa62b P 464343bddb8a42d39d18253f7ddfc32a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c9cb6210ca6432088b94e62e5b395ac in term 1.
I20260619 16:58:54.102821 29356 raft_consensus.cc:2454] T 92b707f03b204711af02d57bdd4fa62b P 3acbb99594704efeab28df68782b198a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c9cb6210ca6432088b94e62e5b395ac in term 1.
I20260619 16:58:54.103025 29010 leader_election.cc:302] T 92b707f03b204711af02d57bdd4fa62b P 5c9cb6210ca6432088b94e62e5b395ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 464343bddb8a42d39d18253f7ddfc32a, 5c9cb6210ca6432088b94e62e5b395ac; no voters:
I20260619 16:58:54.103148 29440 raft_consensus.cc:2781] T 92b707f03b204711af02d57bdd4fa62b P 5c9cb6210ca6432088b94e62e5b395ac [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:54.103238 29440 raft_consensus.cc:686] T 92b707f03b204711af02d57bdd4fa62b P 5c9cb6210ca6432088b94e62e5b395ac [term 1 LEADER]: Becoming Leader. State: Replica: 5c9cb6210ca6432088b94e62e5b395ac, State: Running, Role: LEADER
I20260619 16:58:54.103294 29440 consensus_queue.cc:237] T 92b707f03b204711af02d57bdd4fa62b P 5c9cb6210ca6432088b94e62e5b395ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } }
I20260619 16:58:54.103969 28919 catalog_manager.cc:5495] T 92b707f03b204711af02d57bdd4fa62b P 5c9cb6210ca6432088b94e62e5b395ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c9cb6210ca6432088b94e62e5b395ac (127.20.231.129). New cstate: current_term: 1 leader_uuid: "5c9cb6210ca6432088b94e62e5b395ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } health_report { overall_health: UNKNOWN } } }
Jun 19 16:58:54 dist-test-slave-vnbj krb5kdc[28874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888333, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.190@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM
I20260619 16:58:54.163002 29464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:54.155651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41933 (local address 127.0.0.1:49964)
0619 16:58:54.155878 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:54.155890 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:54.156002 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:54.156173 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:54.156174 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:54.156511 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:54.156558 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:54.156560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:54.157233 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:54.157235 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:54.157786 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:54.157788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:54.158139 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:54.158140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:54.158191 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:54.158831 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:54.158836 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:54.160421 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:54.161885 (+ 1464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:54.161894 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:54.161896 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:54.162089 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:54.162289 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:54.162290 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:54.162290 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:54.162350 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:54.162580 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:54.162587 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:54.162632 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 16:58:54.162714 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 16:58:54.162868 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":5,"thread_start_us":130,"threads_started":1}
I20260619 16:58:54.163002 28969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:54.155795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49964 (local address 127.20.231.190:41933)
0619 16:58:54.155867 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:54.155869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:54.155982 (+ 113us) server_negotiation.cc:407] Connection header received
0619 16:58:54.156066 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:54.156067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:54.156076 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:54.156156 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:54.156661 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:54.157146 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:54.157876 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:54.158060 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:54.158139 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:54.160506 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:54.160508 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:54.160508 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:54.160516 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:54.161744 (+ 1228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:54.162172 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:54.162173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:54.162173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:54.162226 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:54.162408 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:54.162409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:54.162409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:54.162519 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:54.162589 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:54.162734 (+ 145us) server_negotiation.cc:299] Negotiation successful
0619 16:58:54.162922 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":109,"server-negotiator.queue_time_us":43}
I20260619 16:58:54.165407 28919 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:49964:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 16:58:54.165637 28919 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:58:54.167068 29183 tablet_service.cc:1467] Processing CreateTablet for tablet 11280e4374cd40f597ded312291f7ecb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6cd33c6924864226a4646c548eaf898a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 16:58:54.167061 29056 tablet_service.cc:1467] Processing CreateTablet for tablet 11280e4374cd40f597ded312291f7ecb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6cd33c6924864226a4646c548eaf898a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 16:58:54.167223 29056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11280e4374cd40f597ded312291f7ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:54.167223 29183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11280e4374cd40f597ded312291f7ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:54.168691 29430 tablet_bootstrap.cc:492] T 11280e4374cd40f597ded312291f7ecb P 5c9cb6210ca6432088b94e62e5b395ac: Bootstrap starting.
I20260619 16:58:54.168696 29432 tablet_bootstrap.cc:492] T 11280e4374cd40f597ded312291f7ecb P 464343bddb8a42d39d18253f7ddfc32a: Bootstrap starting.
I20260619 16:58:54.168802 29336 tablet_service.cc:1467] Processing CreateTablet for tablet 11280e4374cd40f597ded312291f7ecb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6cd33c6924864226a4646c548eaf898a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 16:58:54.168928 29336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11280e4374cd40f597ded312291f7ecb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:54.169072 29430 tablet_bootstrap.cc:654] T 11280e4374cd40f597ded312291f7ecb P 5c9cb6210ca6432088b94e62e5b395ac: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:54.169127 29432 tablet_bootstrap.cc:654] T 11280e4374cd40f597ded312291f7ecb P 464343bddb8a42d39d18253f7ddfc32a: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:54.169689 29430 tablet_bootstrap.cc:492] T 11280e4374cd40f597ded312291f7ecb P 5c9cb6210ca6432088b94e62e5b395ac: No bootstrap required, opened a new log
I20260619 16:58:54.169734 29430 ts_tablet_manager.cc:1397] T 11280e4374cd40f597ded312291f7ecb P 5c9cb6210ca6432088b94e62e5b395ac: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:54.169871 29430 raft_consensus.cc:348] T 11280e4374cd40f597ded312291f7ecb P 5c9cb6210ca6432088b94e62e5b395ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } }
I20260619 16:58:54.169937 29430 raft_consensus.cc:374] T 11280e4374cd40f597ded312291f7ecb P 5c9cb6210ca6432088b94e62e5b395ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:54.169952 29430 raft_consensus.cc:729] T 11280e4374cd40f597ded312291f7ecb P 5c9cb6210ca6432088b94e62e5b395ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c9cb6210ca6432088b94e62e5b395ac, State: Initialized, Role: FOLLOWER
I20260619 16:58:54.170002 29430 consensus_queue.cc:260] T 11280e4374cd40f597ded312291f7ecb P 5c9cb6210ca6432088b94e62e5b395ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } }
I20260619 16:58:54.170104 29430 ts_tablet_manager.cc:1428] T 11280e4374cd40f597ded312291f7ecb P 5c9cb6210ca6432088b94e62e5b395ac: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:54.170118 29440 tablet_replica.cc:439] T 11280e4374cd40f597ded312291f7ecb P 5c9cb6210ca6432088b94e62e5b395ac: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } } }
I20260619 16:58:54.170202 29440 tablet_replica.cc:442] T 11280e4374cd40f597ded312291f7ecb P 5c9cb6210ca6432088b94e62e5b395ac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:58:54.170282 29431 tablet_bootstrap.cc:492] T 11280e4374cd40f597ded312291f7ecb P 3acbb99594704efeab28df68782b198a: Bootstrap starting.
I20260619 16:58:54.170568 29432 tablet_bootstrap.cc:492] T 11280e4374cd40f597ded312291f7ecb P 464343bddb8a42d39d18253f7ddfc32a: No bootstrap required, opened a new log
I20260619 16:58:54.170612 29432 ts_tablet_manager.cc:1397] T 11280e4374cd40f597ded312291f7ecb P 464343bddb8a42d39d18253f7ddfc32a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:58:54.170652 29431 tablet_bootstrap.cc:654] T 11280e4374cd40f597ded312291f7ecb P 3acbb99594704efeab28df68782b198a: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:54.170773 29432 raft_consensus.cc:348] T 11280e4374cd40f597ded312291f7ecb P 464343bddb8a42d39d18253f7ddfc32a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } }
I20260619 16:58:54.170882 29432 raft_consensus.cc:374] T 11280e4374cd40f597ded312291f7ecb P 464343bddb8a42d39d18253f7ddfc32a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:54.170913 29432 raft_consensus.cc:729] T 11280e4374cd40f597ded312291f7ecb P 464343bddb8a42d39d18253f7ddfc32a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 464343bddb8a42d39d18253f7ddfc32a, State: Initialized, Role: FOLLOWER
I20260619 16:58:54.170961 29432 consensus_queue.cc:260] T 11280e4374cd40f597ded312291f7ecb P 464343bddb8a42d39d18253f7ddfc32a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } }
I20260619 16:58:54.171052 29432 ts_tablet_manager.cc:1428] T 11280e4374cd40f597ded312291f7ecb P 464343bddb8a42d39d18253f7ddfc32a: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260619 16:58:54.171203 29436 tablet_replica.cc:439] T 11280e4374cd40f597ded312291f7ecb P 464343bddb8a42d39d18253f7ddfc32a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } } }
I20260619 16:58:54.171278 29436 tablet_replica.cc:442] T 11280e4374cd40f597ded312291f7ecb P 464343bddb8a42d39d18253f7ddfc32a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:58:54.171509 29431 tablet_bootstrap.cc:492] T 11280e4374cd40f597ded312291f7ecb P 3acbb99594704efeab28df68782b198a: No bootstrap required, opened a new log
I20260619 16:58:54.171561 29431 ts_tablet_manager.cc:1397] T 11280e4374cd40f597ded312291f7ecb P 3acbb99594704efeab28df68782b198a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:54.171694 29431 raft_consensus.cc:348] T 11280e4374cd40f597ded312291f7ecb P 3acbb99594704efeab28df68782b198a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } }
I20260619 16:58:54.171782 29431 raft_consensus.cc:374] T 11280e4374cd40f597ded312291f7ecb P 3acbb99594704efeab28df68782b198a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:54.171820 29431 raft_consensus.cc:729] T 11280e4374cd40f597ded312291f7ecb P 3acbb99594704efeab28df68782b198a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3acbb99594704efeab28df68782b198a, State: Initialized, Role: FOLLOWER
I20260619 16:58:54.171865 29431 consensus_queue.cc:260] T 11280e4374cd40f597ded312291f7ecb P 3acbb99594704efeab28df68782b198a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } }
I20260619 16:58:54.171960 29431 ts_tablet_manager.cc:1428] T 11280e4374cd40f597ded312291f7ecb P 3acbb99594704efeab28df68782b198a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:54.172013 29438 tablet_replica.cc:439] T 11280e4374cd40f597ded312291f7ecb P 3acbb99594704efeab28df68782b198a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } } }
I20260619 16:58:54.172108 29438 tablet_replica.cc:442] T 11280e4374cd40f597ded312291f7ecb P 3acbb99594704efeab28df68782b198a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:58:54.274974 29438 raft_consensus.cc:482] T 11280e4374cd40f597ded312291f7ecb P 3acbb99594704efeab28df68782b198a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:58:54.275071 29438 raft_consensus.cc:504] T 11280e4374cd40f597ded312291f7ecb P 3acbb99594704efeab28df68782b198a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } }
I20260619 16:58:54.275413 29438 leader_election.cc:288] T 11280e4374cd40f597ded312291f7ecb P 3acbb99594704efeab28df68782b198a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c9cb6210ca6432088b94e62e5b395ac (127.20.231.129:46675), 464343bddb8a42d39d18253f7ddfc32a (127.20.231.130:45993)
I20260619 16:58:54.278358 29295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:54.275586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:46675 (local address 127.20.231.131:55491)
0619 16:58:54.275661 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:54.275673 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:54.275721 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:54.275924 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:54.275925 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:54.275927 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:54.275996 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:54.275998 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:54.276638 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:54.276639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:54.277468 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:54.277470 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:54.278079 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:54.278080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:54.278168 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:54.278171 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:58:54.278242 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 16:58:54.278285 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260619 16:58:54.278358 29426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:54.275718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:55491 (local address 127.20.231.129:46675)
0619 16:58:54.275779 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:54.275781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:54.275793 (+ 12us) server_negotiation.cc:407] Connection header received
0619 16:58:54.275827 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:54.275827 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:54.275837 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:54.275918 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:54.276068 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:54.276533 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:54.277565 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:54.277972 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:54.278079 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:54.278088 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:54.278252 (+ 164us) server_negotiation.cc:299] Negotiation successful
0619 16:58:54.278287 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 16:58:54.278628 29428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:54.275706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:42407 (local address 127.20.231.130:45993)
0619 16:58:54.275782 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:54.275823 (+ 41us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:54.275880 (+ 57us) server_negotiation.cc:407] Connection header received
0619 16:58:54.275944 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:54.275946 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:54.275955 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:54.276035 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:58:54.276187 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:54.276645 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:54.277872 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:54.278255 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:54.278338 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:54.278346 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:54.278539 (+ 193us) server_negotiation.cc:299] Negotiation successful
0619 16:58:54.278583 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 16:58:54.278659 29468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:54.275624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:45993 (local address 127.20.231.131:42407)
0619 16:58:54.275832 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:54.275841 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:54.275891 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:54.276041 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:54.276041 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:54.276043 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:58:54.276121 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:54.276122 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:54.276727 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:54.276728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:54.277738 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:54.277740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:54.278337 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:54.278338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:54.278456 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:54.278463 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 16:58:54.278519 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 16:58:54.278570 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":111,"threads_started":1}
I20260619 16:58:54.278753 29076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11280e4374cd40f597ded312291f7ecb" candidate_uuid: "3acbb99594704efeab28df68782b198a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c9cb6210ca6432088b94e62e5b395ac" is_pre_election: true
I20260619 16:58:54.278923 29076 raft_consensus.cc:2454] T 11280e4374cd40f597ded312291f7ecb P 5c9cb6210ca6432088b94e62e5b395ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3acbb99594704efeab28df68782b198a in term 0.
I20260619 16:58:54.278995 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11280e4374cd40f597ded312291f7ecb" candidate_uuid: "3acbb99594704efeab28df68782b198a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "464343bddb8a42d39d18253f7ddfc32a" is_pre_election: true
I20260619 16:58:54.279105 29216 raft_consensus.cc:2454] T 11280e4374cd40f597ded312291f7ecb P 464343bddb8a42d39d18253f7ddfc32a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3acbb99594704efeab28df68782b198a in term 0.
I20260619 16:58:54.279155 29288 leader_election.cc:302] T 11280e4374cd40f597ded312291f7ecb P 3acbb99594704efeab28df68782b198a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3acbb99594704efeab28df68782b198a, 5c9cb6210ca6432088b94e62e5b395ac; no voters:
I20260619 16:58:54.279317 29438 raft_consensus.cc:2781] T 11280e4374cd40f597ded312291f7ecb P 3acbb99594704efeab28df68782b198a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:58:54.279369 29438 raft_consensus.cc:482] T 11280e4374cd40f597ded312291f7ecb P 3acbb99594704efeab28df68782b198a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:58:54.279388 29438 raft_consensus.cc:3037] T 11280e4374cd40f597ded312291f7ecb P 3acbb99594704efeab28df68782b198a [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:54.280117 29438 raft_consensus.cc:504] T 11280e4374cd40f597ded312291f7ecb P 3acbb99594704efeab28df68782b198a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } }
I20260619 16:58:54.280257 29438 leader_election.cc:288] T 11280e4374cd40f597ded312291f7ecb P 3acbb99594704efeab28df68782b198a [CANDIDATE]: Term 1 election: Requested vote from peers 5c9cb6210ca6432088b94e62e5b395ac (127.20.231.129:46675), 464343bddb8a42d39d18253f7ddfc32a (127.20.231.130:45993)
I20260619 16:58:54.280424 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11280e4374cd40f597ded312291f7ecb" candidate_uuid: "3acbb99594704efeab28df68782b198a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "464343bddb8a42d39d18253f7ddfc32a"
I20260619 16:58:54.280503 29216 raft_consensus.cc:3037] T 11280e4374cd40f597ded312291f7ecb P 464343bddb8a42d39d18253f7ddfc32a [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:54.280493 29076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11280e4374cd40f597ded312291f7ecb" candidate_uuid: "3acbb99594704efeab28df68782b198a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c9cb6210ca6432088b94e62e5b395ac"
I20260619 16:58:54.280588 29076 raft_consensus.cc:3037] T 11280e4374cd40f597ded312291f7ecb P 5c9cb6210ca6432088b94e62e5b395ac [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:54.281208 29216 raft_consensus.cc:2454] T 11280e4374cd40f597ded312291f7ecb P 464343bddb8a42d39d18253f7ddfc32a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3acbb99594704efeab28df68782b198a in term 1.
I20260619 16:58:54.281371 29076 raft_consensus.cc:2454] T 11280e4374cd40f597ded312291f7ecb P 5c9cb6210ca6432088b94e62e5b395ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3acbb99594704efeab28df68782b198a in term 1.
I20260619 16:58:54.281399 29290 leader_election.cc:302] T 11280e4374cd40f597ded312291f7ecb P 3acbb99594704efeab28df68782b198a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3acbb99594704efeab28df68782b198a, 464343bddb8a42d39d18253f7ddfc32a; no voters:
I20260619 16:58:54.281570 29438 raft_consensus.cc:2781] T 11280e4374cd40f597ded312291f7ecb P 3acbb99594704efeab28df68782b198a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:54.281742 29438 raft_consensus.cc:686] T 11280e4374cd40f597ded312291f7ecb P 3acbb99594704efeab28df68782b198a [term 1 LEADER]: Becoming Leader. State: Replica: 3acbb99594704efeab28df68782b198a, State: Running, Role: LEADER
I20260619 16:58:54.281836 29438 consensus_queue.cc:237] T 11280e4374cd40f597ded312291f7ecb P 3acbb99594704efeab28df68782b198a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } }
I20260619 16:58:54.282213 29469 tablet_replica.cc:439] T 11280e4374cd40f597ded312291f7ecb P 3acbb99594704efeab28df68782b198a: TxnStatusTablet state changed. Reason: New leader 3acbb99594704efeab28df68782b198a. Latest consensus state: current_term: 1 leader_uuid: "3acbb99594704efeab28df68782b198a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } } }
I20260619 16:58:54.282332 29469 tablet_replica.cc:442] T 11280e4374cd40f597ded312291f7ecb P 3acbb99594704efeab28df68782b198a: This TxnStatusTablet replica's current role is: LEADER
I20260619 16:58:54.282492 29471 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 16:58:54.282591 29471 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 16:58:54.282584 28917 catalog_manager.cc:5495] T 11280e4374cd40f597ded312291f7ecb P 3acbb99594704efeab28df68782b198a reported cstate change: term changed from 0 to 1, leader changed from <none> to 3acbb99594704efeab28df68782b198a (127.20.231.131). New cstate: current_term: 1 leader_uuid: "3acbb99594704efeab28df68782b198a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } health_report { overall_health: HEALTHY } } }
I20260619 16:58:54.288971 29464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:54.286342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:46261 (local address 127.0.0.1:53226)
0619 16:58:54.286417 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:54.286429 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:54.286478 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:54.286636 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:54.286637 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:54.286638 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 16:58:54.286704 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:54.286706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:54.287480 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:54.287481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:54.288052 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:54.288054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:54.288425 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:54.288426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:54.288501 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:54.288520 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 16:58:54.288805 (+ 285us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 16:58:54.288814 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 16:58:54.288854 (+ 40us) client_negotiation.cc:240] Negotiation successful
0619 16:58:54.288903 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260619 16:58:54.288964 29429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:54.286429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53226 (local address 127.20.231.131:46261)
0619 16:58:54.286473 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:54.286475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:54.286485 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:58:54.286557 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:54.286558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:54.286576 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:54.286634 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:58:54.286790 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:54.287399 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:54.288138 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:54.288350 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:54.288425 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:54.288621 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:58:54.288726 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:58:54.288799 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:54.288869 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 16:58:54.288891 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 16:58:54.290061 29216 raft_consensus.cc:1261] T 11280e4374cd40f597ded312291f7ecb P 464343bddb8a42d39d18253f7ddfc32a [term 1 FOLLOWER]: Refusing update from remote peer 3acbb99594704efeab28df68782b198a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:54.290189 29076 raft_consensus.cc:1261] T 11280e4374cd40f597ded312291f7ecb P 5c9cb6210ca6432088b94e62e5b395ac [term 1 FOLLOWER]: Refusing update from remote peer 3acbb99594704efeab28df68782b198a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:54.290423 29469 consensus_queue.cc:1035] T 11280e4374cd40f597ded312291f7ecb P 3acbb99594704efeab28df68782b198a [LEADER]: Connected to new peer: Peer: permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:54.290585 29438 consensus_queue.cc:1035] T 11280e4374cd40f597ded312291f7ecb P 3acbb99594704efeab28df68782b198a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:54.291119 29436 tablet_replica.cc:439] T 11280e4374cd40f597ded312291f7ecb P 464343bddb8a42d39d18253f7ddfc32a: TxnStatusTablet state changed. Reason: New leader 3acbb99594704efeab28df68782b198a. Latest consensus state: current_term: 1 leader_uuid: "3acbb99594704efeab28df68782b198a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } } }
I20260619 16:58:54.291217 29436 tablet_replica.cc:442] T 11280e4374cd40f597ded312291f7ecb P 464343bddb8a42d39d18253f7ddfc32a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:58:54.291479 29440 tablet_replica.cc:439] T 11280e4374cd40f597ded312291f7ecb P 5c9cb6210ca6432088b94e62e5b395ac: TxnStatusTablet state changed. Reason: New leader 3acbb99594704efeab28df68782b198a. Latest consensus state: current_term: 1 leader_uuid: "3acbb99594704efeab28df68782b198a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } } }
I20260619 16:58:54.291570 29440 tablet_replica.cc:442] T 11280e4374cd40f597ded312291f7ecb P 5c9cb6210ca6432088b94e62e5b395ac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:58:54.291734 29469 tablet_replica.cc:439] T 11280e4374cd40f597ded312291f7ecb P 3acbb99594704efeab28df68782b198a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3acbb99594704efeab28df68782b198a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } } }
I20260619 16:58:54.291808 29469 tablet_replica.cc:442] T 11280e4374cd40f597ded312291f7ecb P 3acbb99594704efeab28df68782b198a: This TxnStatusTablet replica's current role is: LEADER
I20260619 16:58:54.292259 29436 tablet_replica.cc:439] T 11280e4374cd40f597ded312291f7ecb P 464343bddb8a42d39d18253f7ddfc32a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3acbb99594704efeab28df68782b198a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } } }
I20260619 16:58:54.292363 29436 tablet_replica.cc:442] T 11280e4374cd40f597ded312291f7ecb P 464343bddb8a42d39d18253f7ddfc32a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:58:54.295588 29438 tablet_replica.cc:439] T 11280e4374cd40f597ded312291f7ecb P 3acbb99594704efeab28df68782b198a: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3acbb99594704efeab28df68782b198a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } } }
I20260619 16:58:54.295686 29438 tablet_replica.cc:442] T 11280e4374cd40f597ded312291f7ecb P 3acbb99594704efeab28df68782b198a: This TxnStatusTablet replica's current role is: LEADER
I20260619 16:58:54.296152 29440 tablet_replica.cc:439] T 11280e4374cd40f597ded312291f7ecb P 5c9cb6210ca6432088b94e62e5b395ac: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3acbb99594704efeab28df68782b198a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } } }
I20260619 16:58:54.296257 29440 tablet_replica.cc:442] T 11280e4374cd40f597ded312291f7ecb P 5c9cb6210ca6432088b94e62e5b395ac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 16:58:54.298166 29478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:54.295197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:46261 (local address 127.20.231.129:39061)
0619 16:58:54.295372 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:54.295386 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:54.295445 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:54.295585 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:54.295586 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:54.295588 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 16:58:54.295636 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:54.295638 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:54.296630 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:54.296631 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:54.297230 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:54.297232 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:54.297713 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:54.297714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:54.297789 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:54.297802 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 16:58:54.298004 (+ 202us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 16:58:54.298007 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:58:54.298061 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 16:58:54.298105 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":99,"threads_started":1}
I20260619 16:58:54.298166 29429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:54.295402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:39061 (local address 127.20.231.131:46261)
0619 16:58:54.295452 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:54.295454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:54.295479 (+ 25us) server_negotiation.cc:407] Connection header received
0619 16:58:54.295500 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:54.295501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:54.295510 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:54.295569 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:58:54.295797 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:54.296485 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:54.297317 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:54.297632 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:54.297710 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:54.297877 (+ 167us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:58:54.297912 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:58:54.297966 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:54.298073 (+ 107us) server_negotiation.cc:299] Negotiation successful
0619 16:58:54.298093 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 16:58:54.300354 29216 raft_consensus.cc:1261] T 92b707f03b204711af02d57bdd4fa62b P 464343bddb8a42d39d18253f7ddfc32a [term 1 FOLLOWER]: Refusing update from remote peer 5c9cb6210ca6432088b94e62e5b395ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:54.300401 29356 raft_consensus.cc:1261] T 92b707f03b204711af02d57bdd4fa62b P 3acbb99594704efeab28df68782b198a [term 1 FOLLOWER]: Refusing update from remote peer 5c9cb6210ca6432088b94e62e5b395ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:54.300622 29440 consensus_queue.cc:1035] T 92b707f03b204711af02d57bdd4fa62b P 5c9cb6210ca6432088b94e62e5b395ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:54.300724 29437 consensus_queue.cc:1035] T 92b707f03b204711af02d57bdd4fa62b P 5c9cb6210ca6432088b94e62e5b395ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:54.309628 29426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:54.307123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:55163 (local address 127.20.231.129:46675)
0619 16:58:54.307159 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:54.307161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:54.307192 (+ 31us) server_negotiation.cc:407] Connection header received
0619 16:58:54.307271 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:54.307271 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:54.307283 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:54.307340 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:58:54.307452 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:54.307914 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:54.309010 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:54.309195 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:54.309249 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:54.309441 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:58:54.309468 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:58:54.309505 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:54.309556 (+ 51us) server_negotiation.cc:299] Negotiation successful
0619 16:58:54.309573 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 16:58:54.309674 29408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:54.307082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:46675 (local address 127.20.231.131:55163)
0619 16:58:54.307150 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:54.307162 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:54.307206 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:54.307340 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:54.307341 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:54.307342 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 16:58:54.307404 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:54.307406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:54.307986 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:54.307987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:54.308931 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:54.308934 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:54.309264 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:54.309265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:54.309339 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:54.309366 (+ 27us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 16:58:54.309506 (+ 140us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 16:58:54.309507 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 16:58:54.309555 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 16:58:54.309597 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 16:58:54.320504 28917 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:58:54.320616 28917 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:58:54.321208 28917 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4616d2b9af7648f9ac7f8bcc6a5e185d: Sending DeleteTablet for 3 replicas of tablet 92b707f03b204711af02d57bdd4fa62b
I20260619 16:58:54.321491 29183 tablet_service.cc:1514] Processing DeleteTablet for tablet 92b707f03b204711af02d57bdd4fa62b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:54 UTC) from {username='oryx', principal='oryx/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:57890
I20260619 16:58:54.321521 29056 tablet_service.cc:1514] Processing DeleteTablet for tablet 92b707f03b204711af02d57bdd4fa62b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:54 UTC) from {username='oryx', principal='oryx/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:46354
I20260619 16:58:54.321586 29452 tablet_replica.cc:330] T 92b707f03b204711af02d57bdd4fa62b P 464343bddb8a42d39d18253f7ddfc32a: stopping tablet replica
I20260619 16:58:54.321630 29452 raft_consensus.cc:2229] T 92b707f03b204711af02d57bdd4fa62b P 464343bddb8a42d39d18253f7ddfc32a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:58:54.321646 29453 tablet_replica.cc:330] T 92b707f03b204711af02d57bdd4fa62b P 5c9cb6210ca6432088b94e62e5b395ac: stopping tablet replica
I20260619 16:58:54.321668 29452 raft_consensus.cc:2258] T 92b707f03b204711af02d57bdd4fa62b P 464343bddb8a42d39d18253f7ddfc32a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:54.321696 29453 raft_consensus.cc:2229] T 92b707f03b204711af02d57bdd4fa62b P 5c9cb6210ca6432088b94e62e5b395ac [term 1 LEADER]: Raft consensus shutting down.
I20260619 16:58:54.321745 29453 raft_consensus.cc:2258] T 92b707f03b204711af02d57bdd4fa62b P 5c9cb6210ca6432088b94e62e5b395ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:54.321911 29336 tablet_service.cc:1514] Processing DeleteTablet for tablet 92b707f03b204711af02d57bdd4fa62b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:54 UTC) from {username='oryx', principal='oryx/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:53222
I20260619 16:58:54.322101 29453 ts_tablet_manager.cc:1905] T 92b707f03b204711af02d57bdd4fa62b P 5c9cb6210ca6432088b94e62e5b395ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:54.322223 29452 ts_tablet_manager.cc:1905] T 92b707f03b204711af02d57bdd4fa62b P 464343bddb8a42d39d18253f7ddfc32a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:54.324000 29454 tablet_replica.cc:330] T 92b707f03b204711af02d57bdd4fa62b P 3acbb99594704efeab28df68782b198a: stopping tablet replica
I20260619 16:58:54.324070 29454 raft_consensus.cc:2229] T 92b707f03b204711af02d57bdd4fa62b P 3acbb99594704efeab28df68782b198a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:58:54.324072 29452 ts_tablet_manager.cc:1918] T 92b707f03b204711af02d57bdd4fa62b P 464343bddb8a42d39d18253f7ddfc32a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 16:58:54.324115 29454 raft_consensus.cc:2258] T 92b707f03b204711af02d57bdd4fa62b P 3acbb99594704efeab28df68782b198a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:54.324127 29452 log.cc:1196] T 92b707f03b204711af02d57bdd4fa62b P 464343bddb8a42d39d18253f7ddfc32a: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-1/wal/wals/92b707f03b204711af02d57bdd4fa62b
I20260619 16:58:54.324365 29452 ts_tablet_manager.cc:1939] T 92b707f03b204711af02d57bdd4fa62b P 464343bddb8a42d39d18253f7ddfc32a: Deleting consensus metadata
I20260619 16:58:54.324700 28906 catalog_manager.cc:4841] TS 464343bddb8a42d39d18253f7ddfc32a (127.20.231.130:45993): tablet 92b707f03b204711af02d57bdd4fa62b (table test-table [id=95de3dc5aee54b17b82c23dc493fdef1]) successfully deleted
I20260619 16:58:54.324749 29453 ts_tablet_manager.cc:1918] T 92b707f03b204711af02d57bdd4fa62b P 5c9cb6210ca6432088b94e62e5b395ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 16:58:54.324801 29453 log.cc:1196] T 92b707f03b204711af02d57bdd4fa62b P 5c9cb6210ca6432088b94e62e5b395ac: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-0/wal/wals/92b707f03b204711af02d57bdd4fa62b
I20260619 16:58:54.325009 29453 ts_tablet_manager.cc:1939] T 92b707f03b204711af02d57bdd4fa62b P 5c9cb6210ca6432088b94e62e5b395ac: Deleting consensus metadata
I20260619 16:58:54.325139 29454 ts_tablet_manager.cc:1905] T 92b707f03b204711af02d57bdd4fa62b P 3acbb99594704efeab28df68782b198a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:54.325315 28904 catalog_manager.cc:4841] TS 5c9cb6210ca6432088b94e62e5b395ac (127.20.231.129:46675): tablet 92b707f03b204711af02d57bdd4fa62b (table test-table [id=95de3dc5aee54b17b82c23dc493fdef1]) successfully deleted
I20260619 16:58:54.326629 29454 ts_tablet_manager.cc:1918] T 92b707f03b204711af02d57bdd4fa62b P 3acbb99594704efeab28df68782b198a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 16:58:54.326676 29454 log.cc:1196] T 92b707f03b204711af02d57bdd4fa62b P 3acbb99594704efeab28df68782b198a: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-2/wal/wals/92b707f03b204711af02d57bdd4fa62b
I20260619 16:58:54.326924 29454 ts_tablet_manager.cc:1939] T 92b707f03b204711af02d57bdd4fa62b P 3acbb99594704efeab28df68782b198a: Deleting consensus metadata
I20260619 16:58:54.327235 28904 catalog_manager.cc:4841] TS 3acbb99594704efeab28df68782b198a (127.20.231.131:46261): tablet 92b707f03b204711af02d57bdd4fa62b (table test-table [id=95de3dc5aee54b17b82c23dc493fdef1]) successfully deleted
Jun 19 16:58:54 dist-test-slave-vnbj krb5kdc[28874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 16:58:54 dist-test-slave-vnbj krb5kdc[28874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:58:54 dist-test-slave-vnbj krb5kdc[28874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
W20260619 16:58:54.353791 28969 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:54.347951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49966 (local address 127.20.231.190:41933)
0619 16:58:54.348012 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:54.348014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:54.348130 (+ 116us) server_negotiation.cc:407] Connection header received
0619 16:58:54.348229 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:54.348230 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:54.348240 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:54.348298 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:54.348901 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:54.349402 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:54.350149 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:54.350354 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:54.350456 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:54.353714 (+ 3258us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49966: BlockingRecv error: recv got EOF from 127.0.0.1:49966 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Jun 19 16:58:54 dist-test-slave-vnbj krb5kdc[28874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM
I20260619 16:58:54.362566 28969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:54.354674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49970 (local address 127.20.231.190:41933)
0619 16:58:54.354711 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:54.354713 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:54.354795 (+ 82us) server_negotiation.cc:407] Connection header received
0619 16:58:54.354896 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:54.354897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:54.354906 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:54.354963 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:54.355547 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:54.356134 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:54.357140 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:54.357322 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:54.357376 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:54.360026 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:54.360028 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:54.360028 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:54.360037 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:54.361442 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:54.361811 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:54.361812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:54.361813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:54.361869 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:54.362055 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:54.362056 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:54.362056 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:54.362174 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:54.362277 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:54.362377 (+ 100us) server_negotiation.cc:299] Negotiation successful
0619 16:58:54.362458 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 16:58:54.364163 28917 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49970:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:58:54.364310 28917 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:58:54.365676 29056 tablet_service.cc:1467] Processing CreateTablet for tablet b606fe66128945ce9e10523f75711686 (DEFAULT_TABLE table=test-table [id=485ffefb027846959478b0dc4912dc48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:54.365742 29183 tablet_service.cc:1467] Processing CreateTablet for tablet b606fe66128945ce9e10523f75711686 (DEFAULT_TABLE table=test-table [id=485ffefb027846959478b0dc4912dc48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:54.365736 29336 tablet_service.cc:1467] Processing CreateTablet for tablet b606fe66128945ce9e10523f75711686 (DEFAULT_TABLE table=test-table [id=485ffefb027846959478b0dc4912dc48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:54.365828 29056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b606fe66128945ce9e10523f75711686. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:54.365846 29183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b606fe66128945ce9e10523f75711686. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:54.365870 29336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b606fe66128945ce9e10523f75711686. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:54.367147 29432 tablet_bootstrap.cc:492] T b606fe66128945ce9e10523f75711686 P 464343bddb8a42d39d18253f7ddfc32a: Bootstrap starting.
I20260619 16:58:54.367522 29432 tablet_bootstrap.cc:654] T b606fe66128945ce9e10523f75711686 P 464343bddb8a42d39d18253f7ddfc32a: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:54.367543 29430 tablet_bootstrap.cc:492] T b606fe66128945ce9e10523f75711686 P 5c9cb6210ca6432088b94e62e5b395ac: Bootstrap starting.
I20260619 16:58:54.367848 29430 tablet_bootstrap.cc:654] T b606fe66128945ce9e10523f75711686 P 5c9cb6210ca6432088b94e62e5b395ac: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:54.368091 29432 tablet_bootstrap.cc:492] T b606fe66128945ce9e10523f75711686 P 464343bddb8a42d39d18253f7ddfc32a: No bootstrap required, opened a new log
I20260619 16:58:54.368134 29432 ts_tablet_manager.cc:1397] T b606fe66128945ce9e10523f75711686 P 464343bddb8a42d39d18253f7ddfc32a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 16:58:54.368294 29432 raft_consensus.cc:348] T b606fe66128945ce9e10523f75711686 P 464343bddb8a42d39d18253f7ddfc32a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } }
I20260619 16:58:54.368364 29432 raft_consensus.cc:374] T b606fe66128945ce9e10523f75711686 P 464343bddb8a42d39d18253f7ddfc32a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:54.368378 29432 raft_consensus.cc:729] T b606fe66128945ce9e10523f75711686 P 464343bddb8a42d39d18253f7ddfc32a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 464343bddb8a42d39d18253f7ddfc32a, State: Initialized, Role: FOLLOWER
I20260619 16:58:54.368433 29430 tablet_bootstrap.cc:492] T b606fe66128945ce9e10523f75711686 P 5c9cb6210ca6432088b94e62e5b395ac: No bootstrap required, opened a new log
I20260619 16:58:54.368434 29432 consensus_queue.cc:260] T b606fe66128945ce9e10523f75711686 P 464343bddb8a42d39d18253f7ddfc32a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } }
I20260619 16:58:54.368476 29430 ts_tablet_manager.cc:1397] T b606fe66128945ce9e10523f75711686 P 5c9cb6210ca6432088b94e62e5b395ac: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:54.368518 29432 ts_tablet_manager.cc:1428] T b606fe66128945ce9e10523f75711686 P 464343bddb8a42d39d18253f7ddfc32a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:54.368577 29430 raft_consensus.cc:348] T b606fe66128945ce9e10523f75711686 P 5c9cb6210ca6432088b94e62e5b395ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } }
I20260619 16:58:54.368639 29430 raft_consensus.cc:374] T b606fe66128945ce9e10523f75711686 P 5c9cb6210ca6432088b94e62e5b395ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:54.368660 29430 raft_consensus.cc:729] T b606fe66128945ce9e10523f75711686 P 5c9cb6210ca6432088b94e62e5b395ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c9cb6210ca6432088b94e62e5b395ac, State: Initialized, Role: FOLLOWER
I20260619 16:58:54.368705 29430 consensus_queue.cc:260] T b606fe66128945ce9e10523f75711686 P 5c9cb6210ca6432088b94e62e5b395ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } }
I20260619 16:58:54.368795 29430 ts_tablet_manager.cc:1428] T b606fe66128945ce9e10523f75711686 P 5c9cb6210ca6432088b94e62e5b395ac: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:54.369659 29431 tablet_bootstrap.cc:492] T b606fe66128945ce9e10523f75711686 P 3acbb99594704efeab28df68782b198a: Bootstrap starting.
I20260619 16:58:54.370007 29431 tablet_bootstrap.cc:654] T b606fe66128945ce9e10523f75711686 P 3acbb99594704efeab28df68782b198a: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:54.370553 29431 tablet_bootstrap.cc:492] T b606fe66128945ce9e10523f75711686 P 3acbb99594704efeab28df68782b198a: No bootstrap required, opened a new log
I20260619 16:58:54.370592 29431 ts_tablet_manager.cc:1397] T b606fe66128945ce9e10523f75711686 P 3acbb99594704efeab28df68782b198a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:54.370703 29431 raft_consensus.cc:348] T b606fe66128945ce9e10523f75711686 P 3acbb99594704efeab28df68782b198a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } }
I20260619 16:58:54.370759 29431 raft_consensus.cc:374] T b606fe66128945ce9e10523f75711686 P 3acbb99594704efeab28df68782b198a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:54.370771 29431 raft_consensus.cc:729] T b606fe66128945ce9e10523f75711686 P 3acbb99594704efeab28df68782b198a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3acbb99594704efeab28df68782b198a, State: Initialized, Role: FOLLOWER
I20260619 16:58:54.370824 29431 consensus_queue.cc:260] T b606fe66128945ce9e10523f75711686 P 3acbb99594704efeab28df68782b198a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } }
I20260619 16:58:54.370977 29431 ts_tablet_manager.cc:1428] T b606fe66128945ce9e10523f75711686 P 3acbb99594704efeab28df68782b198a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:54.405128 29469 raft_consensus.cc:482] T b606fe66128945ce9e10523f75711686 P 3acbb99594704efeab28df68782b198a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:58:54.405277 29469 raft_consensus.cc:504] T b606fe66128945ce9e10523f75711686 P 3acbb99594704efeab28df68782b198a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } }
I20260619 16:58:54.405475 29469 leader_election.cc:288] T b606fe66128945ce9e10523f75711686 P 3acbb99594704efeab28df68782b198a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5c9cb6210ca6432088b94e62e5b395ac (127.20.231.129:46675), 464343bddb8a42d39d18253f7ddfc32a (127.20.231.130:45993)
I20260619 16:58:54.405706 29076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b606fe66128945ce9e10523f75711686" candidate_uuid: "3acbb99594704efeab28df68782b198a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c9cb6210ca6432088b94e62e5b395ac" is_pre_election: true
I20260619 16:58:54.405833 29076 raft_consensus.cc:2454] T b606fe66128945ce9e10523f75711686 P 5c9cb6210ca6432088b94e62e5b395ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3acbb99594704efeab28df68782b198a in term 0.
I20260619 16:58:54.405802 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b606fe66128945ce9e10523f75711686" candidate_uuid: "3acbb99594704efeab28df68782b198a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "464343bddb8a42d39d18253f7ddfc32a" is_pre_election: true
I20260619 16:58:54.405911 29216 raft_consensus.cc:2454] T b606fe66128945ce9e10523f75711686 P 464343bddb8a42d39d18253f7ddfc32a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3acbb99594704efeab28df68782b198a in term 0.
I20260619 16:58:54.406082 29290 leader_election.cc:302] T b606fe66128945ce9e10523f75711686 P 3acbb99594704efeab28df68782b198a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3acbb99594704efeab28df68782b198a, 464343bddb8a42d39d18253f7ddfc32a; no voters:
I20260619 16:58:54.406209 29469 raft_consensus.cc:2781] T b606fe66128945ce9e10523f75711686 P 3acbb99594704efeab28df68782b198a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:58:54.406246 29469 raft_consensus.cc:482] T b606fe66128945ce9e10523f75711686 P 3acbb99594704efeab28df68782b198a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:58:54.406268 29469 raft_consensus.cc:3037] T b606fe66128945ce9e10523f75711686 P 3acbb99594704efeab28df68782b198a [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:54.407087 29469 raft_consensus.cc:504] T b606fe66128945ce9e10523f75711686 P 3acbb99594704efeab28df68782b198a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } }
I20260619 16:58:54.407222 29469 leader_election.cc:288] T b606fe66128945ce9e10523f75711686 P 3acbb99594704efeab28df68782b198a [CANDIDATE]: Term 1 election: Requested vote from peers 5c9cb6210ca6432088b94e62e5b395ac (127.20.231.129:46675), 464343bddb8a42d39d18253f7ddfc32a (127.20.231.130:45993)
I20260619 16:58:54.407361 29076 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b606fe66128945ce9e10523f75711686" candidate_uuid: "3acbb99594704efeab28df68782b198a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c9cb6210ca6432088b94e62e5b395ac"
I20260619 16:58:54.407437 29076 raft_consensus.cc:3037] T b606fe66128945ce9e10523f75711686 P 5c9cb6210ca6432088b94e62e5b395ac [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:54.407424 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b606fe66128945ce9e10523f75711686" candidate_uuid: "3acbb99594704efeab28df68782b198a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "464343bddb8a42d39d18253f7ddfc32a"
I20260619 16:58:54.407500 29216 raft_consensus.cc:3037] T b606fe66128945ce9e10523f75711686 P 464343bddb8a42d39d18253f7ddfc32a [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:54.408070 29216 raft_consensus.cc:2454] T b606fe66128945ce9e10523f75711686 P 464343bddb8a42d39d18253f7ddfc32a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3acbb99594704efeab28df68782b198a in term 1.
I20260619 16:58:54.408260 29076 raft_consensus.cc:2454] T b606fe66128945ce9e10523f75711686 P 5c9cb6210ca6432088b94e62e5b395ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3acbb99594704efeab28df68782b198a in term 1.
I20260619 16:58:54.408331 29290 leader_election.cc:302] T b606fe66128945ce9e10523f75711686 P 3acbb99594704efeab28df68782b198a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3acbb99594704efeab28df68782b198a, 464343bddb8a42d39d18253f7ddfc32a; no voters:
I20260619 16:58:54.408460 29469 raft_consensus.cc:2781] T b606fe66128945ce9e10523f75711686 P 3acbb99594704efeab28df68782b198a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:54.408535 29469 raft_consensus.cc:686] T b606fe66128945ce9e10523f75711686 P 3acbb99594704efeab28df68782b198a [term 1 LEADER]: Becoming Leader. State: Replica: 3acbb99594704efeab28df68782b198a, State: Running, Role: LEADER
I20260619 16:58:54.408601 29469 consensus_queue.cc:237] T b606fe66128945ce9e10523f75711686 P 3acbb99594704efeab28df68782b198a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } }
I20260619 16:58:54.409207 28919 catalog_manager.cc:5495] T b606fe66128945ce9e10523f75711686 P 3acbb99594704efeab28df68782b198a reported cstate change: term changed from 0 to 1, leader changed from <none> to 3acbb99594704efeab28df68782b198a (127.20.231.131). New cstate: current_term: 1 leader_uuid: "3acbb99594704efeab28df68782b198a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } health_report { overall_health: HEALTHY } } }
I20260619 16:58:54.418887 29429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:54.416560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53228 (local address 127.20.231.131:46261)
0619 16:58:54.416618 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:54.416620 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:54.416631 (+ 11us) server_negotiation.cc:407] Connection header received
0619 16:58:54.416769 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:54.416770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:54.416781 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:54.416837 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:58:54.416961 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:54.417433 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:54.418194 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:54.418386 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:54.418458 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:54.418604 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:58:54.418643 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:58:54.418699 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:54.418762 (+ 63us) server_negotiation.cc:299] Negotiation successful
0619 16:58:54.418782 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 16:58:54.419737 29216 raft_consensus.cc:1261] T b606fe66128945ce9e10523f75711686 P 464343bddb8a42d39d18253f7ddfc32a [term 1 FOLLOWER]: Refusing update from remote peer 3acbb99594704efeab28df68782b198a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:54.419817 29076 raft_consensus.cc:1261] T b606fe66128945ce9e10523f75711686 P 5c9cb6210ca6432088b94e62e5b395ac [term 1 FOLLOWER]: Refusing update from remote peer 3acbb99594704efeab28df68782b198a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:54.419961 29469 consensus_queue.cc:1035] T b606fe66128945ce9e10523f75711686 P 3acbb99594704efeab28df68782b198a [LEADER]: Connected to new peer: Peer: permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:54.420081 29469 consensus_queue.cc:1035] T b606fe66128945ce9e10523f75711686 P 3acbb99594704efeab28df68782b198a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:54.423461 28917 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49970:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:58:54.423573 28917 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49970:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:58:54.424172 28917 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4616d2b9af7648f9ac7f8bcc6a5e185d: Sending DeleteTablet for 3 replicas of tablet b606fe66128945ce9e10523f75711686
I20260619 16:58:54.424495 29183 tablet_service.cc:1514] Processing DeleteTablet for tablet b606fe66128945ce9e10523f75711686 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:54 UTC) from {username='oryx', principal='oryx/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:57890
I20260619 16:58:54.424494 29056 tablet_service.cc:1514] Processing DeleteTablet for tablet b606fe66128945ce9e10523f75711686 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:54 UTC) from {username='oryx', principal='oryx/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:46354
I20260619 16:58:54.424584 29452 tablet_replica.cc:330] T b606fe66128945ce9e10523f75711686 P 464343bddb8a42d39d18253f7ddfc32a: stopping tablet replica
I20260619 16:58:54.424583 29453 tablet_replica.cc:330] T b606fe66128945ce9e10523f75711686 P 5c9cb6210ca6432088b94e62e5b395ac: stopping tablet replica
I20260619 16:58:54.424623 29452 raft_consensus.cc:2229] T b606fe66128945ce9e10523f75711686 P 464343bddb8a42d39d18253f7ddfc32a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:58:54.424628 29453 raft_consensus.cc:2229] T b606fe66128945ce9e10523f75711686 P 5c9cb6210ca6432088b94e62e5b395ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:58:54.424670 29453 raft_consensus.cc:2258] T b606fe66128945ce9e10523f75711686 P 5c9cb6210ca6432088b94e62e5b395ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:54.424670 29452 raft_consensus.cc:2258] T b606fe66128945ce9e10523f75711686 P 464343bddb8a42d39d18253f7ddfc32a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:54.424861 29336 tablet_service.cc:1514] Processing DeleteTablet for tablet b606fe66128945ce9e10523f75711686 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:54 UTC) from {username='oryx', principal='oryx/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:53222
I20260619 16:58:54.424942 29454 tablet_replica.cc:330] T b606fe66128945ce9e10523f75711686 P 3acbb99594704efeab28df68782b198a: stopping tablet replica
I20260619 16:58:54.425001 29454 raft_consensus.cc:2229] T b606fe66128945ce9e10523f75711686 P 3acbb99594704efeab28df68782b198a [term 1 LEADER]: Raft consensus shutting down.
I20260619 16:58:54.425092 29454 raft_consensus.cc:2258] T b606fe66128945ce9e10523f75711686 P 3acbb99594704efeab28df68782b198a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:54.425357 29454 ts_tablet_manager.cc:1905] T b606fe66128945ce9e10523f75711686 P 3acbb99594704efeab28df68782b198a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:54.425297 28917 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49970:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:58:54.425438 28917 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:58:54.425768 29453 ts_tablet_manager.cc:1905] T b606fe66128945ce9e10523f75711686 P 5c9cb6210ca6432088b94e62e5b395ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:54.427013 29454 ts_tablet_manager.cc:1918] T b606fe66128945ce9e10523f75711686 P 3acbb99594704efeab28df68782b198a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:58:54.427076 29454 log.cc:1196] T b606fe66128945ce9e10523f75711686 P 3acbb99594704efeab28df68782b198a: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-2/wal/wals/b606fe66128945ce9e10523f75711686
I20260619 16:58:54.427294 29454 ts_tablet_manager.cc:1939] T b606fe66128945ce9e10523f75711686 P 3acbb99594704efeab28df68782b198a: Deleting consensus metadata
I20260619 16:58:54.427417 29453 ts_tablet_manager.cc:1918] T b606fe66128945ce9e10523f75711686 P 5c9cb6210ca6432088b94e62e5b395ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:58:54.427470 29453 log.cc:1196] T b606fe66128945ce9e10523f75711686 P 5c9cb6210ca6432088b94e62e5b395ac: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-0/wal/wals/b606fe66128945ce9e10523f75711686
I20260619 16:58:54.427682 29453 ts_tablet_manager.cc:1939] T b606fe66128945ce9e10523f75711686 P 5c9cb6210ca6432088b94e62e5b395ac: Deleting consensus metadata
I20260619 16:58:54.427903 29452 ts_tablet_manager.cc:1905] T b606fe66128945ce9e10523f75711686 P 464343bddb8a42d39d18253f7ddfc32a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:54.428689 29056 tablet_service.cc:1467] Processing CreateTablet for tablet 6512512ad9664da79a1de558f7632031 (DEFAULT_TABLE table=test-table [id=e8f29f4f3c964ee4931a655347e3710d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:54.428794 29056 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6512512ad9664da79a1de558f7632031. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:54.429102 29452 ts_tablet_manager.cc:1918] T b606fe66128945ce9e10523f75711686 P 464343bddb8a42d39d18253f7ddfc32a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:58:54.429162 29452 log.cc:1196] T b606fe66128945ce9e10523f75711686 P 464343bddb8a42d39d18253f7ddfc32a: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-1/wal/wals/b606fe66128945ce9e10523f75711686
I20260619 16:58:54.429255 28904 catalog_manager.cc:4841] TS 5c9cb6210ca6432088b94e62e5b395ac (127.20.231.129:46675): tablet b606fe66128945ce9e10523f75711686 (table test-table [id=485ffefb027846959478b0dc4912dc48]) successfully deleted
I20260619 16:58:54.429390 29452 ts_tablet_manager.cc:1939] T b606fe66128945ce9e10523f75711686 P 464343bddb8a42d39d18253f7ddfc32a: Deleting consensus metadata
I20260619 16:58:54.429666 29183 tablet_service.cc:1467] Processing CreateTablet for tablet 6512512ad9664da79a1de558f7632031 (DEFAULT_TABLE table=test-table [id=e8f29f4f3c964ee4931a655347e3710d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:54.429766 29183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6512512ad9664da79a1de558f7632031. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:54.430064 29430 tablet_bootstrap.cc:492] T 6512512ad9664da79a1de558f7632031 P 5c9cb6210ca6432088b94e62e5b395ac: Bootstrap starting.
I20260619 16:58:54.430428 29430 tablet_bootstrap.cc:654] T 6512512ad9664da79a1de558f7632031 P 5c9cb6210ca6432088b94e62e5b395ac: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:54.430678 29432 tablet_bootstrap.cc:492] T 6512512ad9664da79a1de558f7632031 P 464343bddb8a42d39d18253f7ddfc32a: Bootstrap starting.
I20260619 16:58:54.430756 28904 catalog_manager.cc:4841] TS 3acbb99594704efeab28df68782b198a (127.20.231.131:46261): tablet b606fe66128945ce9e10523f75711686 (table test-table [id=485ffefb027846959478b0dc4912dc48]) successfully deleted
I20260619 16:58:54.430835 29336 tablet_service.cc:1467] Processing CreateTablet for tablet 6512512ad9664da79a1de558f7632031 (DEFAULT_TABLE table=test-table [id=e8f29f4f3c964ee4931a655347e3710d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:58:54.430970 29336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6512512ad9664da79a1de558f7632031. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:54.431332 29432 tablet_bootstrap.cc:654] T 6512512ad9664da79a1de558f7632031 P 464343bddb8a42d39d18253f7ddfc32a: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:54.431941 29430 tablet_bootstrap.cc:492] T 6512512ad9664da79a1de558f7632031 P 5c9cb6210ca6432088b94e62e5b395ac: No bootstrap required, opened a new log
I20260619 16:58:54.431991 29432 tablet_bootstrap.cc:492] T 6512512ad9664da79a1de558f7632031 P 464343bddb8a42d39d18253f7ddfc32a: No bootstrap required, opened a new log
I20260619 16:58:54.431998 29430 ts_tablet_manager.cc:1397] T 6512512ad9664da79a1de558f7632031 P 5c9cb6210ca6432088b94e62e5b395ac: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 16:58:54.432035 29432 ts_tablet_manager.cc:1397] T 6512512ad9664da79a1de558f7632031 P 464343bddb8a42d39d18253f7ddfc32a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260619 16:58:54.432098 29430 raft_consensus.cc:348] T 6512512ad9664da79a1de558f7632031 P 5c9cb6210ca6432088b94e62e5b395ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } }
I20260619 16:58:54.432169 29430 raft_consensus.cc:374] T 6512512ad9664da79a1de558f7632031 P 5c9cb6210ca6432088b94e62e5b395ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:54.432190 29430 raft_consensus.cc:729] T 6512512ad9664da79a1de558f7632031 P 5c9cb6210ca6432088b94e62e5b395ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c9cb6210ca6432088b94e62e5b395ac, State: Initialized, Role: FOLLOWER
I20260619 16:58:54.432176 29432 raft_consensus.cc:348] T 6512512ad9664da79a1de558f7632031 P 464343bddb8a42d39d18253f7ddfc32a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } }
I20260619 16:58:54.432238 29432 raft_consensus.cc:374] T 6512512ad9664da79a1de558f7632031 P 464343bddb8a42d39d18253f7ddfc32a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:54.432260 29432 raft_consensus.cc:729] T 6512512ad9664da79a1de558f7632031 P 464343bddb8a42d39d18253f7ddfc32a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 464343bddb8a42d39d18253f7ddfc32a, State: Initialized, Role: FOLLOWER
I20260619 16:58:54.432235 29430 consensus_queue.cc:260] T 6512512ad9664da79a1de558f7632031 P 5c9cb6210ca6432088b94e62e5b395ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } }
I20260619 16:58:54.432299 29432 consensus_queue.cc:260] T 6512512ad9664da79a1de558f7632031 P 464343bddb8a42d39d18253f7ddfc32a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } }
I20260619 16:58:54.432355 29430 ts_tablet_manager.cc:1428] T 6512512ad9664da79a1de558f7632031 P 5c9cb6210ca6432088b94e62e5b395ac: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:54.432382 29432 ts_tablet_manager.cc:1428] T 6512512ad9664da79a1de558f7632031 P 464343bddb8a42d39d18253f7ddfc32a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:54.432437 29431 tablet_bootstrap.cc:492] T 6512512ad9664da79a1de558f7632031 P 3acbb99594704efeab28df68782b198a: Bootstrap starting.
I20260619 16:58:54.432677 29431 tablet_bootstrap.cc:654] T 6512512ad9664da79a1de558f7632031 P 3acbb99594704efeab28df68782b198a: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:54.433276 28906 catalog_manager.cc:4841] TS 464343bddb8a42d39d18253f7ddfc32a (127.20.231.130:45993): tablet b606fe66128945ce9e10523f75711686 (table test-table [id=485ffefb027846959478b0dc4912dc48]) successfully deleted
I20260619 16:58:54.433597 29431 tablet_bootstrap.cc:492] T 6512512ad9664da79a1de558f7632031 P 3acbb99594704efeab28df68782b198a: No bootstrap required, opened a new log
I20260619 16:58:54.433638 29431 ts_tablet_manager.cc:1397] T 6512512ad9664da79a1de558f7632031 P 3acbb99594704efeab28df68782b198a: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260619 16:58:54.433708 29431 raft_consensus.cc:348] T 6512512ad9664da79a1de558f7632031 P 3acbb99594704efeab28df68782b198a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } }
I20260619 16:58:54.433759 29431 raft_consensus.cc:374] T 6512512ad9664da79a1de558f7632031 P 3acbb99594704efeab28df68782b198a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:54.433772 29431 raft_consensus.cc:729] T 6512512ad9664da79a1de558f7632031 P 3acbb99594704efeab28df68782b198a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3acbb99594704efeab28df68782b198a, State: Initialized, Role: FOLLOWER
I20260619 16:58:54.433825 29431 consensus_queue.cc:260] T 6512512ad9664da79a1de558f7632031 P 3acbb99594704efeab28df68782b198a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } }
I20260619 16:58:54.433917 29431 ts_tablet_manager.cc:1428] T 6512512ad9664da79a1de558f7632031 P 3acbb99594704efeab28df68782b198a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:54.565968 29440 raft_consensus.cc:482] T 6512512ad9664da79a1de558f7632031 P 5c9cb6210ca6432088b94e62e5b395ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:58:54.566076 29440 raft_consensus.cc:504] T 6512512ad9664da79a1de558f7632031 P 5c9cb6210ca6432088b94e62e5b395ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } }
I20260619 16:58:54.566280 29440 leader_election.cc:288] T 6512512ad9664da79a1de558f7632031 P 5c9cb6210ca6432088b94e62e5b395ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 464343bddb8a42d39d18253f7ddfc32a (127.20.231.130:45993), 3acbb99594704efeab28df68782b198a (127.20.231.131:46261)
I20260619 16:58:54.566538 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6512512ad9664da79a1de558f7632031" candidate_uuid: "5c9cb6210ca6432088b94e62e5b395ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "464343bddb8a42d39d18253f7ddfc32a" is_pre_election: true
I20260619 16:58:54.566629 29216 raft_consensus.cc:2454] T 6512512ad9664da79a1de558f7632031 P 464343bddb8a42d39d18253f7ddfc32a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c9cb6210ca6432088b94e62e5b395ac in term 0.
I20260619 16:58:54.566565 29356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6512512ad9664da79a1de558f7632031" candidate_uuid: "5c9cb6210ca6432088b94e62e5b395ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3acbb99594704efeab28df68782b198a" is_pre_election: true
I20260619 16:58:54.566777 29356 raft_consensus.cc:2454] T 6512512ad9664da79a1de558f7632031 P 3acbb99594704efeab28df68782b198a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c9cb6210ca6432088b94e62e5b395ac in term 0.
I20260619 16:58:54.566799 29010 leader_election.cc:302] T 6512512ad9664da79a1de558f7632031 P 5c9cb6210ca6432088b94e62e5b395ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 464343bddb8a42d39d18253f7ddfc32a, 5c9cb6210ca6432088b94e62e5b395ac; no voters:
I20260619 16:58:54.566920 29440 raft_consensus.cc:2781] T 6512512ad9664da79a1de558f7632031 P 5c9cb6210ca6432088b94e62e5b395ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:58:54.566952 29440 raft_consensus.cc:482] T 6512512ad9664da79a1de558f7632031 P 5c9cb6210ca6432088b94e62e5b395ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:58:54.566983 29440 raft_consensus.cc:3037] T 6512512ad9664da79a1de558f7632031 P 5c9cb6210ca6432088b94e62e5b395ac [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:54.567713 29440 raft_consensus.cc:504] T 6512512ad9664da79a1de558f7632031 P 5c9cb6210ca6432088b94e62e5b395ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } }
I20260619 16:58:54.567847 29440 leader_election.cc:288] T 6512512ad9664da79a1de558f7632031 P 5c9cb6210ca6432088b94e62e5b395ac [CANDIDATE]: Term 1 election: Requested vote from peers 464343bddb8a42d39d18253f7ddfc32a (127.20.231.130:45993), 3acbb99594704efeab28df68782b198a (127.20.231.131:46261)
I20260619 16:58:54.568005 29216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6512512ad9664da79a1de558f7632031" candidate_uuid: "5c9cb6210ca6432088b94e62e5b395ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "464343bddb8a42d39d18253f7ddfc32a"
I20260619 16:58:54.568030 29356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6512512ad9664da79a1de558f7632031" candidate_uuid: "5c9cb6210ca6432088b94e62e5b395ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3acbb99594704efeab28df68782b198a"
I20260619 16:58:54.568104 29356 raft_consensus.cc:3037] T 6512512ad9664da79a1de558f7632031 P 3acbb99594704efeab28df68782b198a [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:54.568106 29216 raft_consensus.cc:3037] T 6512512ad9664da79a1de558f7632031 P 464343bddb8a42d39d18253f7ddfc32a [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:54.568706 29356 raft_consensus.cc:2454] T 6512512ad9664da79a1de558f7632031 P 3acbb99594704efeab28df68782b198a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c9cb6210ca6432088b94e62e5b395ac in term 1.
I20260619 16:58:54.568727 29216 raft_consensus.cc:2454] T 6512512ad9664da79a1de558f7632031 P 464343bddb8a42d39d18253f7ddfc32a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c9cb6210ca6432088b94e62e5b395ac in term 1.
I20260619 16:58:54.568874 29008 leader_election.cc:302] T 6512512ad9664da79a1de558f7632031 P 5c9cb6210ca6432088b94e62e5b395ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3acbb99594704efeab28df68782b198a, 5c9cb6210ca6432088b94e62e5b395ac; no voters:
I20260619 16:58:54.568981 29440 raft_consensus.cc:2781] T 6512512ad9664da79a1de558f7632031 P 5c9cb6210ca6432088b94e62e5b395ac [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:54.569046 29440 raft_consensus.cc:686] T 6512512ad9664da79a1de558f7632031 P 5c9cb6210ca6432088b94e62e5b395ac [term 1 LEADER]: Becoming Leader. State: Replica: 5c9cb6210ca6432088b94e62e5b395ac, State: Running, Role: LEADER
I20260619 16:58:54.569099 29440 consensus_queue.cc:237] T 6512512ad9664da79a1de558f7632031 P 5c9cb6210ca6432088b94e62e5b395ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } }
I20260619 16:58:54.569734 28917 catalog_manager.cc:5495] T 6512512ad9664da79a1de558f7632031 P 5c9cb6210ca6432088b94e62e5b395ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c9cb6210ca6432088b94e62e5b395ac (127.20.231.129). New cstate: current_term: 1 leader_uuid: "5c9cb6210ca6432088b94e62e5b395ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c9cb6210ca6432088b94e62e5b395ac" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 46675 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 } health_report { overall_health: UNKNOWN } } }
I20260619 16:58:54.581466 29426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:54.578977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46372 (local address 127.20.231.129:46675)
0619 16:58:54.579036 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:54.579038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:54.579080 (+ 42us) server_negotiation.cc:407] Connection header received
0619 16:58:54.579169 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:54.579170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:54.579183 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:54.579250 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:58:54.579379 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:54.579862 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:54.580763 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:54.580963 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:54.581041 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:54.581187 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:58:54.581225 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:58:54.581280 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:54.581341 (+ 61us) server_negotiation.cc:299] Negotiation successful
0619 16:58:54.581365 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 16:58:54.583818 29216 raft_consensus.cc:1261] T 6512512ad9664da79a1de558f7632031 P 464343bddb8a42d39d18253f7ddfc32a [term 1 FOLLOWER]: Refusing update from remote peer 5c9cb6210ca6432088b94e62e5b395ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:54.583856 29356 raft_consensus.cc:1261] T 6512512ad9664da79a1de558f7632031 P 3acbb99594704efeab28df68782b198a [term 1 FOLLOWER]: Refusing update from remote peer 5c9cb6210ca6432088b94e62e5b395ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:58:54.584064 29485 consensus_queue.cc:1035] T 6512512ad9664da79a1de558f7632031 P 5c9cb6210ca6432088b94e62e5b395ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "464343bddb8a42d39d18253f7ddfc32a" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45993 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:54.584185 29440 consensus_queue.cc:1035] T 6512512ad9664da79a1de558f7632031 P 5c9cb6210ca6432088b94e62e5b395ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "3acbb99594704efeab28df68782b198a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 46261 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:58:54.593998 28917 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49970:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:58:54.594154 28917 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49970:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:58:54.594949 28917 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4616d2b9af7648f9ac7f8bcc6a5e185d: Sending DeleteTablet for 3 replicas of tablet 6512512ad9664da79a1de558f7632031
I20260619 16:58:54.595237 29056 tablet_service.cc:1514] Processing DeleteTablet for tablet 6512512ad9664da79a1de558f7632031 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:54 UTC) from {username='oryx', principal='oryx/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:46354
I20260619 16:58:54.595278 29336 tablet_service.cc:1514] Processing DeleteTablet for tablet 6512512ad9664da79a1de558f7632031 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:54 UTC) from {username='oryx', principal='oryx/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:53222
I20260619 16:58:54.595293 29183 tablet_service.cc:1514] Processing DeleteTablet for tablet 6512512ad9664da79a1de558f7632031 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:58:54 UTC) from {username='oryx', principal='oryx/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:57890
I20260619 16:58:54.595332 29453 tablet_replica.cc:330] T 6512512ad9664da79a1de558f7632031 P 5c9cb6210ca6432088b94e62e5b395ac: stopping tablet replica
I20260619 16:58:54.595389 29453 raft_consensus.cc:2229] T 6512512ad9664da79a1de558f7632031 P 5c9cb6210ca6432088b94e62e5b395ac [term 1 LEADER]: Raft consensus shutting down.
I20260619 16:58:54.595403 29452 tablet_replica.cc:330] T 6512512ad9664da79a1de558f7632031 P 464343bddb8a42d39d18253f7ddfc32a: stopping tablet replica
I20260619 16:58:54.595444 29453 raft_consensus.cc:2258] T 6512512ad9664da79a1de558f7632031 P 5c9cb6210ca6432088b94e62e5b395ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:54.595433 29454 tablet_replica.cc:330] T 6512512ad9664da79a1de558f7632031 P 3acbb99594704efeab28df68782b198a: stopping tablet replica
I20260619 16:58:54.595453 29452 raft_consensus.cc:2229] T 6512512ad9664da79a1de558f7632031 P 464343bddb8a42d39d18253f7ddfc32a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:58:54.595492 29452 raft_consensus.cc:2258] T 6512512ad9664da79a1de558f7632031 P 464343bddb8a42d39d18253f7ddfc32a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:54.595499 29454 raft_consensus.cc:2229] T 6512512ad9664da79a1de558f7632031 P 3acbb99594704efeab28df68782b198a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:58:54.595541 29454 raft_consensus.cc:2258] T 6512512ad9664da79a1de558f7632031 P 3acbb99594704efeab28df68782b198a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:58:54.595804 29453 ts_tablet_manager.cc:1905] T 6512512ad9664da79a1de558f7632031 P 5c9cb6210ca6432088b94e62e5b395ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:54.595937 29454 ts_tablet_manager.cc:1905] T 6512512ad9664da79a1de558f7632031 P 3acbb99594704efeab28df68782b198a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:54.595984 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 28991
I20260619 16:58:54.597597 29452 ts_tablet_manager.cc:1905] T 6512512ad9664da79a1de558f7632031 P 464343bddb8a42d39d18253f7ddfc32a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:58:54.597632 29454 ts_tablet_manager.cc:1918] T 6512512ad9664da79a1de558f7632031 P 3acbb99594704efeab28df68782b198a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 16:58:54.597716 29454 log.cc:1196] T 6512512ad9664da79a1de558f7632031 P 3acbb99594704efeab28df68782b198a: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-2/wal/wals/6512512ad9664da79a1de558f7632031
I20260619 16:58:54.597913 29454 ts_tablet_manager.cc:1939] T 6512512ad9664da79a1de558f7632031 P 3acbb99594704efeab28df68782b198a: Deleting consensus metadata
I20260619 16:58:54.598457 28904 catalog_manager.cc:4841] TS 3acbb99594704efeab28df68782b198a (127.20.231.131:46261): tablet 6512512ad9664da79a1de558f7632031 (table test-table [id=e8f29f4f3c964ee4931a655347e3710d]) successfully deleted
I20260619 16:58:54.598997 29452 ts_tablet_manager.cc:1918] T 6512512ad9664da79a1de558f7632031 P 464343bddb8a42d39d18253f7ddfc32a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 16:58:54.599072 29452 log.cc:1196] T 6512512ad9664da79a1de558f7632031 P 464343bddb8a42d39d18253f7ddfc32a: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781888301345144-21406-0/minicluster-data/ts-1/wal/wals/6512512ad9664da79a1de558f7632031
I20260619 16:58:54.599300 29452 ts_tablet_manager.cc:1939] T 6512512ad9664da79a1de558f7632031 P 464343bddb8a42d39d18253f7ddfc32a: Deleting consensus metadata
I20260619 16:58:54.599750 28906 catalog_manager.cc:4841] TS 464343bddb8a42d39d18253f7ddfc32a (127.20.231.130:45993): tablet 6512512ad9664da79a1de558f7632031 (table test-table [id=e8f29f4f3c964ee4931a655347e3710d]) successfully deleted
W20260619 16:58:54.603703 28904 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.231.129:46675 (error 108)
I20260619 16:58:54.603803 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 29131
I20260619 16:58:54.607088 29512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:54.604131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:46675 (local address 127.0.0.1:46382)
0619 16:58:54.606978 (+ 2847us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:54.607035 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.129:46675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2825,"thread_start_us":38,"threads_started":1}
W20260619 16:58:54.607270 28904 catalog_manager.cc:4584] TS 5c9cb6210ca6432088b94e62e5b395ac (127.20.231.129:46675): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6512512ad9664da79a1de558f7632031: Network error: Client connection negotiation failed: client connection to 127.20.231.129:46675: connect: Connection refused (error 111)
I20260619 16:58:54.609005 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 29271
I20260619 16:58:54.614967 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 28887
2026-06-19T16:58:54Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2625 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:58:54 dist-test-slave-vnbj krb5kdc[29515](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:58:54 dist-test-slave-vnbj krb5kdc[29515](info): set up 2 sockets
Jun 19 16:58:54 dist-test-slave-vnbj krb5kdc[29515](info): commencing operation
krb5kdc: starting...
W20260619 16:58:55.671556 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 16:58:55 dist-test-slave-vnbj krb5kdc[29515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888335, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:58:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:58:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.231.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
I20260619 16:58:55.797525 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:40563
--webserver_interface=127.20.231.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:36083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:40563
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:55.879900 29528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:55.880128 29528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:55.880151 29528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:55.881392 29528 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:55.881443 29528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:55.881457 29528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:55.881470 29528 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:55.881484 29528 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:58:55.883040 29528 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:36083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:40563
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:40563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:55.883303 29528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:55.883534 29528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:55.888548 29536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:55.888623 29533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:55.888839 29528 server_base.cc:1034] running on GCE node
W20260619 16:58:55.888548 29534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:55.889382 29528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:55.889698 29528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:55.890897 29528 hybrid_clock.cc:648] HybridClock initialized: now 1781888335890884 us; error 34 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-06-19T16:58:55Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1279 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:58:55 dist-test-slave-vnbj krb5kdc[29541](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:58:55 dist-test-slave-vnbj krb5kdc[29541](info): set up 2 sockets
Jun 19 16:58:55 dist-test-slave-vnbj krb5kdc[29541](info): commencing operation
krb5kdc: starting...
W20260619 16:58:56.943554 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 16:58:57 dist-test-slave-vnbj krb5kdc[29541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:58:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:58:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.231.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
I20260619 16:58:57.062711 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:41629
--webserver_interface=127.20.231.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:34283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:41629
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:57.144299 29554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:57.144516 29554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:57.144538 29554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:57.145675 29554 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:57.145712 29554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:57.145727 29554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:57.145740 29554 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:57.145751 29554 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:58:57.147250 29554 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:34283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:41629
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:41629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:57.147455 29554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:57.147637 29554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:57.152815 29562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:57.152985 29554 server_base.cc:1034] running on GCE node
W20260619 16:58:57.152853 29560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:57.152815 29559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:57.153345 29554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:57.153642 29554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:57.154836 29554 hybrid_clock.cc:648] HybridClock initialized: now 1781888337154817 us; error 38 us; skew 500 ppm
Jun 19 16:58:57 dist-test-slave-vnbj krb5kdc[29541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888337, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:57.157449 29554 init.cc:377] Logged in from keytab as kudu/127.20.231.190@KRBTEST.COM (short username kudu)
I20260619 16:58:57.157948 29554 webserver.cc:460] Webserver started at http://127.20.231.190:33729/ using document root <none> and password file <none>
I20260619 16:58:57.158172 29554 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:57.158238 29554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:57.158383 29554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:57.159562 29554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "5a4c045a6ae0419ba7e2a5f63bc805bc"
format_stamp: "Formatted at 2026-06-19 16:58:57 on dist-test-slave-vnbj"
server_key: "8b23010721d8e4ee91fded38da4cd7f4"
server_key_iv: "6f5ac273d8a16c91060fc651c1b187bb"
server_key_version: "encryptionkey@0"
I20260619 16:58:57.159974 29554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "5a4c045a6ae0419ba7e2a5f63bc805bc"
format_stamp: "Formatted at 2026-06-19 16:58:57 on dist-test-slave-vnbj"
server_key: "8b23010721d8e4ee91fded38da4cd7f4"
server_key_iv: "6f5ac273d8a16c91060fc651c1b187bb"
server_key_version: "encryptionkey@0"
I20260619 16:58:57.161366 29554 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:58:57.162206 29569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:57.162403 29554 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:58:57.162498 29554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "5a4c045a6ae0419ba7e2a5f63bc805bc"
format_stamp: "Formatted at 2026-06-19 16:58:57 on dist-test-slave-vnbj"
server_key: "8b23010721d8e4ee91fded38da4cd7f4"
server_key_iv: "6f5ac273d8a16c91060fc651c1b187bb"
server_key_version: "encryptionkey@0"
I20260619 16:58:57.162580 29554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:57.177080 29554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:57.179486 29554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:57.179661 29554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:57.183797 29554 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:41629
I20260619 16:58:57.183845 29621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:41629 every 8 connection(s)
I20260619 16:58:57.184232 29554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 16:58:57.185118 29622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:58:57.187619 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 29554
I20260619 16:58:57.187685 29622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc: Bootstrap starting.
I20260619 16:58:57.187739 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 16:58:57.187932 21406 external_mini_cluster.cc:1428] Setting key a1092b2d0bf2cec4bbd7c712f066fdde
I20260619 16:58:57.188386 29622 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc: Neither blocks nor log segments found. Creating new log.
I20260619 16:58:57.188751 29622 log.cc:824] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:57.189484 29622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc: No bootstrap required, opened a new log
I20260619 16:58:57.191331 29622 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a4c045a6ae0419ba7e2a5f63bc805bc" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41629 } }
I20260619 16:58:57.191479 29622 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:58:57.191525 29622 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a4c045a6ae0419ba7e2a5f63bc805bc, State: Initialized, Role: FOLLOWER
I20260619 16:58:57.191664 29622 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a4c045a6ae0419ba7e2a5f63bc805bc" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41629 } }
I20260619 16:58:57.191741 29622 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:58:57.191783 29622 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:58:57.191830 29622 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:58:57.192379 29622 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a4c045a6ae0419ba7e2a5f63bc805bc" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41629 } }
I20260619 16:58:57.192489 29622 leader_election.cc:302] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a4c045a6ae0419ba7e2a5f63bc805bc; no voters:
I20260619 16:58:57.192675 29622 leader_election.cc:288] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:58:57.192700 29627 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:58:57.192821 29627 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc [term 1 LEADER]: Becoming Leader. State: Replica: 5a4c045a6ae0419ba7e2a5f63bc805bc, State: Running, Role: LEADER
I20260619 16:58:57.192904 29627 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a4c045a6ae0419ba7e2a5f63bc805bc" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41629 } }
I20260619 16:58:57.192930 29622 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:57.193207 29627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a4c045a6ae0419ba7e2a5f63bc805bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a4c045a6ae0419ba7e2a5f63bc805bc" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41629 } } }
I20260619 16:58:57.193274 29627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:57.193396 29627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a4c045a6ae0419ba7e2a5f63bc805bc. Latest consensus state: current_term: 1 leader_uuid: "5a4c045a6ae0419ba7e2a5f63bc805bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a4c045a6ae0419ba7e2a5f63bc805bc" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41629 } } }
I20260619 16:58:57.193466 29627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:57.193578 29632 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:58:57.194156 29632 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Jun 19 16:58:57 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:57.196561 29632 catalog_manager.cc:1347] Generated new cluster ID: 1fc54ecf182e4287a58a982807c042fd
I20260619 16:58:57.196620 29632 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:58:57.199241 29625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:57.188476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46006 (local address 127.20.231.190:41629)
0619 16:58:57.188671 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:57.188684 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:57.188710 (+ 26us) server_negotiation.cc:407] Connection header received
0619 16:58:57.189323 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:57.189331 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:57.189392 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:57.189539 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:57.190163 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:57.191146 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:57.192085 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:57.192522 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:57.192624 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:57.195487 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:57.195491 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:57.195492 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:57.195504 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:57.197611 (+ 2107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:57.197992 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:57.197995 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:57.197995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:57.198106 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:57.198298 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:57.198299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:57.198300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:57.198613 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:57.198786 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:57.198887 (+ 101us) server_negotiation.cc:299] Negotiation successful
0619 16:58:57.199030 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":84,"threads_started":1}
I20260619 16:58:57.204421 29632 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:58:57.204910 29632 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:58:57.210727 29632 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc: Generated new TSK 0
I20260619 16:58:57.210953 29632 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.231.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab.
Entry for principal kudu/127.20.231.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab.
I20260619 16:58:57.237510 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.129:0
--local_ip_for_outbound_sockets=127.20.231.129
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:41629
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:34283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:57.323995 29648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:57.324277 29648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:57.324324 29648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:57.325724 29648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:57.325801 29648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:57.325851 29648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.129
I20260619 16:58:57.327518 29648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:34283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:41629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29648
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.129
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:57.327817 29648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:57.328071 29648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:57.333565 29656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:57.333545 29654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:57.333545 29653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:57.333989 29648 server_base.cc:1034] running on GCE node
I20260619 16:58:57.334244 29648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:57.334494 29648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:57.335652 29648 hybrid_clock.cc:648] HybridClock initialized: now 1781888337335655 us; error 44 us; skew 500 ppm
Jun 19 16:58:57 dist-test-slave-vnbj krb5kdc[29541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888337, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:57.338333 29648 init.cc:377] Logged in from keytab as kudu/127.20.231.129@KRBTEST.COM (short username kudu)
I20260619 16:58:57.338744 29648 webserver.cc:460] Webserver started at http://127.20.231.129:34897/ using document root <none> and password file <none>
I20260619 16:58:57.339025 29648 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:57.339074 29648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:57.339228 29648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:57.340301 29648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-0/data/instance:
uuid: "a4202ece3f72492aa75b54618b36f538"
format_stamp: "Formatted at 2026-06-19 16:58:57 on dist-test-slave-vnbj"
server_key: "29627b635324eef60ed8fc169e148ee3"
server_key_iv: "a2ef65ef5a8027f15e852d0bbc6d2c72"
server_key_version: "encryptionkey@0"
I20260619 16:58:57.340657 29648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance:
uuid: "a4202ece3f72492aa75b54618b36f538"
format_stamp: "Formatted at 2026-06-19 16:58:57 on dist-test-slave-vnbj"
server_key: "29627b635324eef60ed8fc169e148ee3"
server_key_iv: "a2ef65ef5a8027f15e852d0bbc6d2c72"
server_key_version: "encryptionkey@0"
I20260619 16:58:57.342048 29648 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:58:57.343006 29663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:57.343176 29648 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:57.343271 29648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-0/wal
uuid: "a4202ece3f72492aa75b54618b36f538"
format_stamp: "Formatted at 2026-06-19 16:58:57 on dist-test-slave-vnbj"
server_key: "29627b635324eef60ed8fc169e148ee3"
server_key_iv: "a2ef65ef5a8027f15e852d0bbc6d2c72"
server_key_version: "encryptionkey@0"
I20260619 16:58:57.343350 29648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:57.357532 29648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:57.359936 29648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:57.360064 29648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:57.360262 29648 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:58:57.360589 29648 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:57.360625 29648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:57.360667 29648 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:57.360692 29648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:57.366706 29648 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.129:46565
I20260619 16:58:57.366786 29776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.129:46565 every 8 connection(s)
I20260619 16:58:57.367112 29648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
I20260619 16:58:57.373369 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 29648
I20260619 16:58:57.373481 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance
Jun 19 16:58:57 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888337, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:57.373672 21406 external_mini_cluster.cc:1428] Setting key 03485149790ec4dc24f2d63cb43ea4c9
I20260619 16:58:57.377429 29625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:57.367911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:39909 (local address 127.20.231.190:41629)
0619 16:58:57.367991 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:57.367993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:57.368171 (+ 178us) server_negotiation.cc:407] Connection header received
0619 16:58:57.368706 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:57.368708 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:57.368721 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:57.368774 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:57.369761 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:57.370298 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:57.371398 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:57.371623 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:57.371697 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:57.374137 (+ 2440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:57.374138 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:57.374138 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:57.374145 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:57.375827 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:57.376390 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:57.376392 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:57.376392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:57.376449 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:57.376742 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:57.376743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:57.376743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:57.376858 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:57.376913 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:57.377279 (+ 366us) server_negotiation.cc:299] Negotiation successful
0619 16:58:57.377361 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 16:58:57.377511 29779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:57.367840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.129:39909)
0619 16:58:57.368041 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:57.368088 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:57.368588 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:57.368847 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:57.368851 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:57.369457 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:57.369655 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:57.369660 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:57.370393 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:57.370394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:57.371299 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:57.371301 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:57.371715 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:57.371716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:57.371844 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:57.372379 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:57.372387 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:57.374051 (+ 1664us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:57.375930 (+ 1879us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:57.375939 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:57.375941 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:57.376301 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:57.376518 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:57.376518 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:57.376519 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:57.376679 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:57.376921 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:57.376927 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:57.377109 (+ 182us) client_negotiation.cc:769] Sending connection context
0619 16:58:57.377212 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 16:58:57.377347 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":81,"threads_started":1}
I20260619 16:58:57.378088 29777 heartbeater.cc:347] Connected to a master server at 127.20.231.190:41629
I20260619 16:58:57.378193 29777 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:57.378391 29777 heartbeater.cc:511] Master 127.20.231.190:41629 requested a full tablet report, sending...
I20260619 16:58:57.379115 29586 ts_manager.cc:194] Registered new tserver with Master: a4202ece3f72492aa75b54618b36f538 (127.20.231.129:46565)
I20260619 16:58:57.379824 29586 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.129@KRBTEST.COM'} at 127.20.231.129:39909
WARNING: no policy specified for kudu/127.20.231.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab.
Entry for principal kudu/127.20.231.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab.
I20260619 16:58:57.400004 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.130:0
--local_ip_for_outbound_sockets=127.20.231.130
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:41629
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:34283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:57.483042 29782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:57.483273 29782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:57.483297 29782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:57.484458 29782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:57.484499 29782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:57.484525 29782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.130
I20260619 16:58:57.485968 29782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:34283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:41629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.130
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:57.486223 29782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:57.486460 29782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 16:58:57.491716 29782 server_base.cc:1034] running on GCE node
W20260619 16:58:57.491699 29790 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:57.491815 29788 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:57.491699 29787 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:57.492218 29782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:57.492437 29782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:57.493628 29782 hybrid_clock.cc:648] HybridClock initialized: now 1781888337493600 us; error 46 us; skew 500 ppm
Jun 19 16:58:57 dist-test-slave-vnbj krb5kdc[29541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888337, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:57.496461 29782 init.cc:377] Logged in from keytab as kudu/127.20.231.130@KRBTEST.COM (short username kudu)
I20260619 16:58:57.496939 29782 webserver.cc:460] Webserver started at http://127.20.231.130:36767/ using document root <none> and password file <none>
I20260619 16:58:57.497126 29782 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:57.497166 29782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:57.497260 29782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:57.498283 29782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-1/data/instance:
uuid: "cf117d2fbe4e404788a36aff8a85bef8"
format_stamp: "Formatted at 2026-06-19 16:58:57 on dist-test-slave-vnbj"
server_key: "b7efbecfc831ea9a17102a3712e1252d"
server_key_iv: "1b3254311ec076c11c512cd24ee38750"
server_key_version: "encryptionkey@0"
I20260619 16:58:57.498584 29782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance:
uuid: "cf117d2fbe4e404788a36aff8a85bef8"
format_stamp: "Formatted at 2026-06-19 16:58:57 on dist-test-slave-vnbj"
server_key: "b7efbecfc831ea9a17102a3712e1252d"
server_key_iv: "1b3254311ec076c11c512cd24ee38750"
server_key_version: "encryptionkey@0"
I20260619 16:58:57.499982 29782 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:58:57.500687 29797 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:57.500877 29782 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:58:57.500972 29782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-1/wal
uuid: "cf117d2fbe4e404788a36aff8a85bef8"
format_stamp: "Formatted at 2026-06-19 16:58:57 on dist-test-slave-vnbj"
server_key: "b7efbecfc831ea9a17102a3712e1252d"
server_key_iv: "1b3254311ec076c11c512cd24ee38750"
server_key_version: "encryptionkey@0"
I20260619 16:58:57.501039 29782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:57.513273 29782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:57.515440 29782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:57.515559 29782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:57.515744 29782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:58:57.516076 29782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:57.516129 29782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:57.516171 29782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:57.516196 29782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:57.522091 29782 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.130:45513
I20260619 16:58:57.522177 29910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.130:45513 every 8 connection(s)
I20260619 16:58:57.522496 29782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
I20260619 16:58:57.525218 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 29782
I20260619 16:58:57.525339 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance
I20260619 16:58:57.525497 21406 external_mini_cluster.cc:1428] Setting key 9dc594e5e21bc0b03d3a001d38cb0f07
Jun 19 16:58:57 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888337, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:57.533557 29625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:57.523373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:43299 (local address 127.20.231.190:41629)
0619 16:58:57.523430 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:57.523432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:57.523554 (+ 122us) server_negotiation.cc:407] Connection header received
0619 16:58:57.524076 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:57.524077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:57.524087 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:57.524158 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:57.525190 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:57.525869 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:57.527027 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:57.527357 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:57.527436 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:57.530149 (+ 2713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:57.530150 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:57.530151 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:57.530159 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:57.531988 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:57.532395 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:57.532396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:57.532396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:57.532462 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:57.532840 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:57.532841 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:57.532841 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:57.532989 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:57.533070 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:57.533365 (+ 295us) server_negotiation.cc:299] Negotiation successful
0619 16:58:57.533456 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 16:58:57.533665 29913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:57.523231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.130:43299)
0619 16:58:57.523418 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:57.523470 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:57.523982 (+ 512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:57.524226 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:57.524229 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:57.524834 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:57.525081 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:57.525088 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:57.525957 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:57.525958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:57.526950 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:57.526952 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:57.527437 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:57.527438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:57.527538 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:57.528130 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:57.528141 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:57.530053 (+ 1912us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:57.532088 (+ 2035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:57.532094 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:57.532096 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:57.532324 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:57.532563 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:57.532564 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:57.532564 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:57.532753 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:57.533080 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:57.533086 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:57.533217 (+ 131us) client_negotiation.cc:769] Sending connection context
0619 16:58:57.533319 (+ 102us) client_negotiation.cc:240] Negotiation successful
0619 16:58:57.533449 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":120,"threads_started":1}
I20260619 16:58:57.534197 29911 heartbeater.cc:347] Connected to a master server at 127.20.231.190:41629
I20260619 16:58:57.534300 29911 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:57.534471 29911 heartbeater.cc:511] Master 127.20.231.190:41629 requested a full tablet report, sending...
I20260619 16:58:57.534931 29586 ts_manager.cc:194] Registered new tserver with Master: cf117d2fbe4e404788a36aff8a85bef8 (127.20.231.130:45513)
WARNING: no policy specified for kudu/127.20.231.131@KRBTEST.COM; defaulting to no policy
I20260619 16:58:57.535444 29586 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.130@KRBTEST.COM'} at 127.20.231.130:43299
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab.
Entry for principal kudu/127.20.231.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab.
I20260619 16:58:57.553720 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.131:0
--local_ip_for_outbound_sockets=127.20.231.131
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:41629
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:34283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:57.639937 29916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:57.640163 29916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:57.640192 29916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:57.641377 29916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:57.641417 29916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:57.641444 29916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.131
I20260619 16:58:57.643072 29916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:34283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:41629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.131
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:57.643285 29916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:57.643463 29916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:57.648721 29921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:57.648880 29922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:57.648947 29916 server_base.cc:1034] running on GCE node
W20260619 16:58:57.648749 29924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:57.649196 29916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:57.649452 29916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:57.650602 29916 hybrid_clock.cc:648] HybridClock initialized: now 1781888337650585 us; error 32 us; skew 500 ppm
Jun 19 16:58:57 dist-test-slave-vnbj krb5kdc[29541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888337, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:57.653278 29916 init.cc:377] Logged in from keytab as kudu/127.20.231.131@KRBTEST.COM (short username kudu)
I20260619 16:58:57.653755 29916 webserver.cc:460] Webserver started at http://127.20.231.131:44343/ using document root <none> and password file <none>
I20260619 16:58:57.653980 29916 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:57.654033 29916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:57.654189 29916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:58:57.655332 29916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-2/data/instance:
uuid: "31f6125abd364deeb7f4aa3f2d0479cc"
format_stamp: "Formatted at 2026-06-19 16:58:57 on dist-test-slave-vnbj"
server_key: "31c85b468192f53bf7148fff5d4be66b"
server_key_iv: "d4782c0687cdd60820cb14de3e86ed25"
server_key_version: "encryptionkey@0"
I20260619 16:58:57.655683 29916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance:
uuid: "31f6125abd364deeb7f4aa3f2d0479cc"
format_stamp: "Formatted at 2026-06-19 16:58:57 on dist-test-slave-vnbj"
server_key: "31c85b468192f53bf7148fff5d4be66b"
server_key_iv: "d4782c0687cdd60820cb14de3e86ed25"
server_key_version: "encryptionkey@0"
I20260619 16:58:57.657017 29916 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:58:57.657869 29931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:57.658066 29916 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:57.658190 29916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-2/wal
uuid: "31f6125abd364deeb7f4aa3f2d0479cc"
format_stamp: "Formatted at 2026-06-19 16:58:57 on dist-test-slave-vnbj"
server_key: "31c85b468192f53bf7148fff5d4be66b"
server_key_iv: "d4782c0687cdd60820cb14de3e86ed25"
server_key_version: "encryptionkey@0"
I20260619 16:58:57.658268 29916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:57.681542 29916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:57.683727 29916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:57.683897 29916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:57.684139 29916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:58:57.684494 29916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:57.684547 29916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:57.684582 29916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:57.684609 29916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:57.690658 29916 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.131:33303
I20260619 16:58:57.690800 30044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.131:33303 every 8 connection(s)
I20260619 16:58:57.691119 29916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
Jun 19 16:58:57 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888337, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:58:57.698907 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 29916
I20260619 16:58:57.699019 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance
I20260619 16:58:57.699232 21406 external_mini_cluster.cc:1428] Setting key 1be2716cabb8df11dd3ea5d57761cc41
I20260619 16:58:57.701416 29625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:57.691865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:60063 (local address 127.20.231.190:41629)
0619 16:58:57.691946 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:57.691948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:57.692116 (+ 168us) server_negotiation.cc:407] Connection header received
0619 16:58:57.692612 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:57.692614 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:57.692626 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:57.692687 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:57.693774 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:57.694346 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:57.695452 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:57.695662 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:57.695743 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:57.698050 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:57.698052 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:57.698052 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:57.698061 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:57.700057 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:57.700500 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:57.700501 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:57.700501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:57.700561 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:57.700792 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:57.700793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:57.700793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:57.700928 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:57.700992 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:57.701279 (+ 287us) server_negotiation.cc:299] Negotiation successful
0619 16:58:57.701350 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 16:58:57.701607 30047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:57.691830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.131:60063)
0619 16:58:57.691995 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:57.692031 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:57.692516 (+ 485us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:57.692742 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:57.692746 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:57.693390 (+ 644us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:57.693616 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:57.693622 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:57.694443 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:57.694444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:57.695333 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:57.695335 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:57.695754 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:57.695755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:57.695894 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:57.696372 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:57.696381 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:57.697954 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:58:57.700127 (+ 2173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:57.700135 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:57.700138 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:57.700413 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:57.700624 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:58:57.700625 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:58:57.700625 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:58:57.700735 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:58:57.700985 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:58:57.700989 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:58:57.701112 (+ 123us) client_negotiation.cc:769] Sending connection context
0619 16:58:57.701240 (+ 128us) client_negotiation.cc:240] Negotiation successful
0619 16:58:57.701373 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":99,"threads_started":1}
I20260619 16:58:57.702157 30045 heartbeater.cc:347] Connected to a master server at 127.20.231.190:41629
I20260619 16:58:57.702246 30045 heartbeater.cc:464] Registering TS with master...
I20260619 16:58:57.702399 30045 heartbeater.cc:511] Master 127.20.231.190:41629 requested a full tablet report, sending...
I20260619 16:58:57.702734 29586 ts_manager.cc:194] Registered new tserver with Master: 31f6125abd364deeb7f4aa3f2d0479cc (127.20.231.131:33303)
I20260619 16:58:57.703123 29586 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.131@KRBTEST.COM'} at 127.20.231.131:60063
I20260619 16:58:57.711486 21406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.20.231.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.231.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.231.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.129.keytab.
Entry for principal oryx/127.20.231.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.129.keytab.
WARNING: no policy specified for oryx/127.20.231.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.231.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.231.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.130.keytab.
Entry for principal oryx/127.20.231.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.130.keytab.
WARNING: no policy specified for oryx/127.20.231.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.20.231.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.20.231.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.131.keytab.
Entry for principal oryx/127.20.231.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.131.keytab.
I20260619 16:58:57.790808 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 29648
I20260619 16:58:57.795636 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 29782
I20260619 16:58:57.799847 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 29916
I20260619 16:58:57.804600 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 29554
I20260619 16:58:57.809509 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:41629
--webserver_interface=127.20.231.190
--webserver_port=33729
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:34283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:41629
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:58:57.890053 30054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:57.890298 30054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:57.890323 30054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:57.891592 30054 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:58:57.891637 30054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:57.891652 30054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:57.891665 30054 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:58:57.891677 30054 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:58:57.893245 30054 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:34283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:41629
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:41629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=33729
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:57.893517 30054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:57.893760 30054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:57.898603 30060 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:57.898619 30059 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:57.898610 30062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:57.899085 30054 server_base.cc:1034] running on GCE node
I20260619 16:58:57.899338 30054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:57.899613 30054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:57.900774 30054 hybrid_clock.cc:648] HybridClock initialized: now 1781888337900761 us; error 40 us; skew 500 ppm
Jun 19 16:58:57 dist-test-slave-vnbj krb5kdc[29541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888337, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:57.903232 30054 init.cc:377] Logged in from keytab as kudu/127.20.231.190@KRBTEST.COM (short username kudu)
I20260619 16:58:57.903700 30054 webserver.cc:460] Webserver started at http://127.20.231.190:33729/ using document root <none> and password file <none>
I20260619 16:58:57.903929 30054 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:57.904008 30054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:57.905403 30054 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260619 16:58:57.906046 30069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:57.906204 30054 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:58:57.906308 30054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "5a4c045a6ae0419ba7e2a5f63bc805bc"
format_stamp: "Formatted at 2026-06-19 16:58:57 on dist-test-slave-vnbj"
server_key: "8b23010721d8e4ee91fded38da4cd7f4"
server_key_iv: "6f5ac273d8a16c91060fc651c1b187bb"
server_key_version: "encryptionkey@0"
I20260619 16:58:57.906646 30054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:57.933343 30054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:57.935483 30054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:57.935653 30054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:57.939888 30054 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:41629
I20260619 16:58:57.939919 30121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:41629 every 8 connection(s)
I20260619 16:58:57.940307 30054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 16:58:57.941792 30122 sys_catalog.cc:263] Verifying existing consensus state
I20260619 16:58:57.942420 30122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc: Bootstrap starting.
I20260619 16:58:57.944372 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 30054
I20260619 16:58:57.944659 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.129:46565
--local_ip_for_outbound_sockets=127.20.231.129
--tserver_master_addrs=127.20.231.190:41629
--webserver_port=34897
--webserver_interface=127.20.231.129
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:34283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.20.231.129
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 16:58:57.951234 30122 log.cc:824] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc: Log is configured to *not* fsync() on all Append() calls
I20260619 16:58:57.953011 30122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 16:58:57.953318 30122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc: Bootstrap complete.
I20260619 16:58:57.955695 30122 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a4c045a6ae0419ba7e2a5f63bc805bc" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41629 } }
I20260619 16:58:57.955928 30122 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a4c045a6ae0419ba7e2a5f63bc805bc, State: Initialized, Role: FOLLOWER
I20260619 16:58:57.956097 30122 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a4c045a6ae0419ba7e2a5f63bc805bc" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41629 } }
I20260619 16:58:57.956169 30122 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:58:57.956234 30122 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:58:57.956282 30122 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc [term 1 FOLLOWER]: Advancing to term 2
I20260619 16:58:57.957480 30122 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a4c045a6ae0419ba7e2a5f63bc805bc" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41629 } }
I20260619 16:58:57.957602 30122 leader_election.cc:302] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a4c045a6ae0419ba7e2a5f63bc805bc; no voters:
I20260619 16:58:57.957818 30122 leader_election.cc:288] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 16:58:57.957852 30126 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc [term 2 FOLLOWER]: Leader election won for term 2
I20260619 16:58:57.957979 30126 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc [term 2 LEADER]: Becoming Leader. State: Replica: 5a4c045a6ae0419ba7e2a5f63bc805bc, State: Running, Role: LEADER
I20260619 16:58:57.958107 30122 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:58:57.958083 30126 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a4c045a6ae0419ba7e2a5f63bc805bc" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41629 } }
I20260619 16:58:57.958390 30126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a4c045a6ae0419ba7e2a5f63bc805bc. Latest consensus state: current_term: 2 leader_uuid: "5a4c045a6ae0419ba7e2a5f63bc805bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a4c045a6ae0419ba7e2a5f63bc805bc" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41629 } } }
I20260619 16:58:57.958467 30126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:57.958393 30127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5a4c045a6ae0419ba7e2a5f63bc805bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a4c045a6ae0419ba7e2a5f63bc805bc" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41629 } } }
I20260619 16:58:57.958618 30127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc [sys.catalog]: This master's current role is: LEADER
I20260619 16:58:57.958770 30130 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:58:57.959338 30130 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:58:57.959511 30130 catalog_manager.cc:1259] Loaded cluster ID: 1fc54ecf182e4287a58a982807c042fd
I20260619 16:58:57.959554 30130 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:58:57.960757 30130 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:58:57.960937 30130 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5a4c045a6ae0419ba7e2a5f63bc805bc: Loaded TSK: 0
I20260619 16:58:57.961154 30130 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 16:58:58.030215 30124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:58.030445 30124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:58.030485 30124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:58.031709 30124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:58.031771 30124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:58.031823 30124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.129
I20260619 16:58:58.033239 30124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:34283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.129.keytab
--principal=oryx/127.20.231.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.129:46565
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.231.129
--webserver_port=34897
--tserver_master_addrs=127.20.231.190:41629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.129
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:58.033504 30124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:58.033726 30124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:58.039685 30151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:58.039700 30148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:58.039705 30149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:58.039727 30124 server_base.cc:1034] running on GCE node
I20260619 16:58:58.040143 30124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:58.040407 30124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:58.041556 30124 hybrid_clock.cc:648] HybridClock initialized: now 1781888338041538 us; error 36 us; skew 500 ppm
Jun 19 16:58:58 dist-test-slave-vnbj krb5kdc[29541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888338, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:58.044112 30124 init.cc:377] Logged in from keytab as oryx/127.20.231.129@KRBTEST.COM (short username oryx)
I20260619 16:58:58.044617 30124 webserver.cc:460] Webserver started at http://127.20.231.129:34897/ using document root <none> and password file <none>
I20260619 16:58:58.044865 30124 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:58.044924 30124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:58.046299 30124 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:58.046932 30158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:58.047072 30124 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:58:58.047156 30124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-0/wal
uuid: "a4202ece3f72492aa75b54618b36f538"
format_stamp: "Formatted at 2026-06-19 16:58:57 on dist-test-slave-vnbj"
server_key: "29627b635324eef60ed8fc169e148ee3"
server_key_iv: "a2ef65ef5a8027f15e852d0bbc6d2c72"
server_key_version: "encryptionkey@0"
I20260619 16:58:58.047498 30124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:58.066048 30124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:58.068365 30124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:58.068511 30124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:58.068782 30124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:58:58.069164 30124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:58.069201 30124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:58.069226 30124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:58.069238 30124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:58.076646 30124 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.129:46565
I20260619 16:58:58.076684 30271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.129:46565 every 8 connection(s)
I20260619 16:58:58.077034 30124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
I20260619 16:58:58.080518 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 30124
I20260619 16:58:58.080808 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.130:45513
--local_ip_for_outbound_sockets=127.20.231.130
--tserver_master_addrs=127.20.231.190:41629
--webserver_port=36767
--webserver_interface=127.20.231.130
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:34283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.20.231.130
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 16:58:58 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:58:58 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:58:58.086566 30274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:58.077722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.129:43595)
0619 16:58:58.077926 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:58.077976 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:58.078668 (+ 692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:58.079352 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:58.079356 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:58.079936 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:58.080132 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:58.080137 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.081916 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.081917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:58.082771 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:58.082773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.083424 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.083424 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:58.083559 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:58.084105 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:58.084113 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:58.086341 (+ 2228us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":95,"threads_started":1}
W20260619 16:58:58.086624 30275 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:58.077854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:43595 (local address 127.20.231.190:41629)
0619 16:58:58.078916 (+ 1062us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:58.078925 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:58.078965 (+ 40us) server_negotiation.cc:407] Connection header received
0619 16:58:58.079092 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:58.079098 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:58.079152 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:58.079294 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:58.080284 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.081783 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.082923 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.083325 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.083459 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:58.086476 (+ 3017us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.129:43595: BlockingRecv error: recv got EOF from 127.20.231.129:43595 (error 108)
Metrics: {"server-negotiator.queue_time_us":1012,"thread_start_us":70,"threads_started":1}
W20260619 16:58:58.086794 30272 heartbeater.cc:644] Failed to heartbeat to 127.20.231.190:41629 (0 consecutive failures): Not authorized: Failed to ping master at 127.20.231.190:41629: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Jun 19 16:58:58 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:58:58 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:58:58.091190 30274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:58.087041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.129:49489)
0619 16:58:58.087085 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:58.087105 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:58.087138 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:58.087266 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:58.087267 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:58.087406 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:58.087449 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:58.087450 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.088100 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.088100 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:58.088606 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:58.088607 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.088958 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.088959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:58.089022 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:58.089445 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:58.089449 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:58.091105 (+ 1656us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 16:58:58.091244 30275 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:58.087085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:49489 (local address 127.20.231.190:41629)
0619 16:58:58.087138 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:58.087140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:58.087156 (+ 16us) server_negotiation.cc:407] Connection header received
0619 16:58:58.087181 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:58.087181 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:58.087190 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:58.087262 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:58.087534 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.088004 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.088685 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.088880 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.088950 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:58.091179 (+ 2229us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.129:49489: BlockingRecv error: recv got EOF from 127.20.231.129:49489 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Jun 19 16:58:58 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:58:58 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:58:58.095803 30274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:58.091683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.129:46185)
0619 16:58:58.091770 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:58.091780 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:58.091823 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:58.091961 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:58.091961 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:58.092082 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:58.092107 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:58.092108 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.092700 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.092701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:58.093194 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:58.093195 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.093538 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.093538 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:58.093589 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:58.093997 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:58.094003 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:58.095717 (+ 1714us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":72}
W20260619 16:58:58.095853 30275 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:58.091732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:46185 (local address 127.20.231.190:41629)
0619 16:58:58.091768 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:58.091770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:58.091808 (+ 38us) server_negotiation.cc:407] Connection header received
0619 16:58:58.091880 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:58.091881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:58.091891 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:58.091959 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:58.092184 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.092628 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.093270 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.093458 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.093537 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:58.095784 (+ 2247us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.129:46185: BlockingRecv error: recv got EOF from 127.20.231.129:46185 (error 108)
Metrics: {"server-negotiator.queue_time_us":15}
W20260619 16:58:58.095956 30272 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 16:58:58.167706 30276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:58.167930 30276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:58.167955 30276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:58.169214 30276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:58.169263 30276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:58.169292 30276 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.130
I20260619 16:58:58.170969 30276 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:34283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.130.keytab
--principal=oryx/127.20.231.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.130:45513
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.231.130
--webserver_port=36767
--tserver_master_addrs=127.20.231.190:41629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30276
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.130
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:58.171236 30276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:58.171474 30276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:58:58.176977 30284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:58.176999 30282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:58.176977 30281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:58.177318 30276 server_base.cc:1034] running on GCE node
I20260619 16:58:58.177531 30276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:58.177779 30276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:58.178946 30276 hybrid_clock.cc:648] HybridClock initialized: now 1781888338178923 us; error 38 us; skew 500 ppm
Jun 19 16:58:58 dist-test-slave-vnbj krb5kdc[29541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888338, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:58.181485 30276 init.cc:377] Logged in from keytab as oryx/127.20.231.130@KRBTEST.COM (short username oryx)
I20260619 16:58:58.182039 30276 webserver.cc:460] Webserver started at http://127.20.231.130:36767/ using document root <none> and password file <none>
I20260619 16:58:58.182293 30276 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:58.182365 30276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:58.183835 30276 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:58.184412 30291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:58.184584 30276 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:58:58.184675 30276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-1/wal
uuid: "cf117d2fbe4e404788a36aff8a85bef8"
format_stamp: "Formatted at 2026-06-19 16:58:57 on dist-test-slave-vnbj"
server_key: "b7efbecfc831ea9a17102a3712e1252d"
server_key_iv: "1b3254311ec076c11c512cd24ee38750"
server_key_version: "encryptionkey@0"
I20260619 16:58:58.185072 30276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:58.208356 30276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:58.210570 30276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:58.210727 30276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:58.210984 30276 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:58:58.211306 30276 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:58.211359 30276 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:58.211401 30276 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:58.211436 30276 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:58.217339 30276 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.130:45513
I20260619 16:58:58.217386 30404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.130:45513 every 8 connection(s)
I20260619 16:58:58.217747 30276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
Jun 19 16:58:58 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:58:58 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
W20260619 16:58:58.225605 30275 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:58.218450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:45641 (local address 127.20.231.190:41629)
0619 16:58:58.218500 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:58.218502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:58.218796 (+ 294us) server_negotiation.cc:407] Connection header received
0619 16:58:58.219343 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:58.219344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:58.219358 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:58.219409 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:58.220349 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.220947 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.222150 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.222375 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.222466 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:58.225542 (+ 3076us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.130:45641: BlockingRecv error: recv got EOF from 127.20.231.130:45641 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 16:58:58.225692 30407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:58.218459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.130:45641)
0619 16:58:58.218640 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:58.218698 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:58.219255 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:58.219463 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:58.219466 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:58.220037 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:58.220241 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:58.220246 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.221043 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.221044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:58.222038 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:58.222040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.222475 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.222476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:58.222608 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:58.223217 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:58.223225 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:58.225465 (+ 2240us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":88,"threads_started":1}
W20260619 16:58:58.225982 30405 heartbeater.cc:644] Failed to heartbeat to 127.20.231.190:41629 (0 consecutive failures): Not authorized: Failed to ping master at 127.20.231.190:41629: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
I20260619 16:58:58.226785 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 30276
I20260619 16:58:58.227094 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.131:33303
--local_ip_for_outbound_sockets=127.20.231.131
--tserver_master_addrs=127.20.231.190:41629
--webserver_port=44343
--webserver_interface=127.20.231.131
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:34283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.20.231.131
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 16:58:58 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:58:58 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:58:58.231395 30407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:58.226222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.130:50741)
0619 16:58:58.226276 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:58.226289 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:58.226341 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:58.226454 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:58.226454 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:58.226586 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:58.226651 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:58.226653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.227469 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.227470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:58.228336 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:58.228338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.228957 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.228957 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:58.229003 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:58.229481 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:58.229485 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:58.231338 (+ 1853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 16:58:58.231542 30275 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:58.226277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:50741 (local address 127.20.231.190:41629)
0619 16:58:58.226337 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:58.226339 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:58.226347 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:58:58.226386 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:58.226386 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:58.226395 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:58.226443 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:58.226729 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.227394 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.228429 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.228870 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.228960 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:58.231452 (+ 2492us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.130:50741: BlockingRecv error: recv got EOF from 127.20.231.130:50741 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Jun 19 16:58:58 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:58:58 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:58:58.237007 30407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:58.231863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.130:58757)
0619 16:58:58.231978 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:58.231990 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:58.232046 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:58.232193 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:58.232194 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:58.232360 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:58.232406 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:58.232408 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.233090 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.233091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:58.233984 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:58.233987 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.234357 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.234358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:58.234424 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:58.234968 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:58.234974 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:58.236913 (+ 1939us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":97}
W20260619 16:58:58.237064 30275 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:58.231931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:58757 (local address 127.20.231.190:41629)
0619 16:58:58.231987 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:58.231989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:58.232030 (+ 41us) server_negotiation.cc:407] Connection header received
0619 16:58:58.232106 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:58.232107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:58.232117 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:58.232172 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:58.232481 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.233010 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.234086 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.234282 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.234341 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:58.236994 (+ 2653us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.130:58757: BlockingRecv error: recv got EOF from 127.20.231.130:58757 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260619 16:58:58.237162 30405 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 16:58:58.310820 30408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:58:58.311079 30408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:58:58.311103 30408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:58:58.312377 30408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:58:58.312424 30408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:58:58.312479 30408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.131
I20260619 16:58:58.314158 30408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:34283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/krb5kdc/oryx_127.20.231.131.keytab
--principal=oryx/127.20.231.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.131:33303
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.231.131
--webserver_port=44343
--tserver_master_addrs=127.20.231.190:41629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.131
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:58:58.314404 30408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:58:58.314631 30408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 16:58:58.319860 30408 server_base.cc:1034] running on GCE node
W20260619 16:58:58.319841 30413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:58.319841 30414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:58:58.319829 30416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:58:58.320231 30408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:58:58.320513 30408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:58:58.321666 30408 hybrid_clock.cc:648] HybridClock initialized: now 1781888338321645 us; error 40 us; skew 500 ppm
Jun 19 16:58:58 dist-test-slave-vnbj krb5kdc[29541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888338, etypes {rep=17 tkt=17 ses=17}, oryx/127.20.231.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:58:58.329447 30408 init.cc:377] Logged in from keytab as oryx/127.20.231.131@KRBTEST.COM (short username oryx)
I20260619 16:58:58.329967 30408 webserver.cc:460] Webserver started at http://127.20.231.131:44343/ using document root <none> and password file <none>
I20260619 16:58:58.330210 30408 fs_manager.cc:359] Metadata directory not provided
I20260619 16:58:58.330303 30408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:58:58.331734 30408 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:58:58.332383 30423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:58.332551 30408 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:58:58.332638 30408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-2/wal
uuid: "31f6125abd364deeb7f4aa3f2d0479cc"
format_stamp: "Formatted at 2026-06-19 16:58:57 on dist-test-slave-vnbj"
server_key: "31c85b468192f53bf7148fff5d4be66b"
server_key_iv: "d4782c0687cdd60820cb14de3e86ed25"
server_key_version: "encryptionkey@0"
I20260619 16:58:58.333046 30408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:58:58.348009 30408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:58:58.350492 30408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:58:58.350661 30408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:58:58.350966 30408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:58:58.351286 30408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:58:58.351338 30408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:58.351378 30408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:58:58.351402 30408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:58:58.357271 30408 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.131:33303
I20260619 16:58:58.357319 30536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.131:33303 every 8 connection(s)
I20260619 16:58:58.357689 30408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
I20260619 16:58:58.363131 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 30408
Jun 19 16:58:58 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:58:58 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
W20260619 16:58:58.365576 30275 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:58.358417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:38889 (local address 127.20.231.190:41629)
0619 16:58:58.358463 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:58.358465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:58.358712 (+ 247us) server_negotiation.cc:407] Connection header received
0619 16:58:58.359334 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:58.359336 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:58.359348 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:58.359399 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:58.360349 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.360854 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.361857 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.362060 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.362139 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:58.365515 (+ 3376us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.131:38889: BlockingRecv error: recv got EOF from 127.20.231.131:38889 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 16:58:58.365589 30539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:58.358436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.131:38889)
0619 16:58:58.358595 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:58.358630 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:58.359208 (+ 578us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:58.359478 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:58.359482 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:58.360042 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:58.360231 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:58.360236 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.360946 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.360947 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:58.361755 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:58.361758 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.362148 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.362149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:58.362285 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:58.362889 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:58.362899 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:58.365401 (+ 2502us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":68,"threads_started":1}
W20260619 16:58:58.365847 30537 heartbeater.cc:644] Failed to heartbeat to 127.20.231.190:41629 (0 consecutive failures): Not authorized: Failed to ping master at 127.20.231.190:41629: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Jun 19 16:58:58 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:58:58 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:58:58.370381 30539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:58.366069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.131:47303)
0619 16:58:58.366098 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:58.366116 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:58.366157 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:58.366275 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:58.366276 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:58.366395 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:58.366456 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:58.366458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.367204 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.367205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:58.367760 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:58.367762 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.368213 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.368214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:58.368284 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:58.368687 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:58.368691 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:58.370301 (+ 1610us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":14}
W20260619 16:58:58.370430 30275 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:58.366103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:47303 (local address 127.20.231.190:41629)
0619 16:58:58.366144 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:58.366146 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:58.366155 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:58:58.366216 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:58.366217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:58.366226 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:58.366272 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:58.366531 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.367135 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.367839 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.368150 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.368212 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:58.370366 (+ 2154us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.131:47303: BlockingRecv error: recv got EOF from 127.20.231.131:47303 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 16:58:58.371711 30541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:58.363712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37500 (local address 127.20.231.190:41629)
0619 16:58:58.363845 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:58.363847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:58.363854 (+ 7us) server_negotiation.cc:407] Connection header received
0619 16:58:58.363874 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:58.363874 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:58.363885 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:58.363957 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:58.364665 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.365395 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.366453 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.366702 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.366749 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:58.368325 (+ 1576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:58:58.368329 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:58:58.368329 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:58:58.368339 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:58:58.370148 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:58.370489 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:58.370493 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:58.370493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:58.370600 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:58:58.370953 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:58:58.370954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:58:58.370954 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:58:58.371272 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:58:58.371409 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0619 16:58:58.371521 (+ 112us) server_negotiation.cc:299] Negotiation successful
0619 16:58:58.371602 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":59,"threads_started":1}
Jun 19 16:58:58 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:58:58 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:58:58.375101 30539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:58.370904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.131:58775)
0619 16:58:58.370938 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:58.370948 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:58.370984 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:58.371099 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:58.371100 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:58.371237 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:58.371268 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:58.371269 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.372108 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.372109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:58.372799 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:58.372801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.373102 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.373103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:58.373139 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:58.373445 (+ 306us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:58.373448 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:58.375059 (+ 1611us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":18}
W20260619 16:58:58.375221 30537 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 16:58:58.375250 30275 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:58.370930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:58775 (local address 127.20.231.190:41629)
0619 16:58:58.370975 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:58.370977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:58.370986 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:58:58.371037 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:58.371038 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:58.371045 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:58.371086 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:58.371340 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.372033 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.372869 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:58.373041 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:58.373094 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:58.375152 (+ 2058us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.131:58775: BlockingRecv error: recv got EOF from 127.20.231.131:58775 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Jun 19 16:58:59 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:58:59 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:58:59.100925 30542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:59.096335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.129:60933)
0619 16:58:59.096474 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:59.096485 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:59.096582 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:59.096846 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:59.096847 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:59.096998 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:59.097045 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:59.097047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:59.097714 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:59.097715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:59.098301 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:59.098303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:59.098635 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:59.098635 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:59.098673 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:59.099153 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:59.099158 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:59.100857 (+ 1699us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":83,"threads_started":1}
W20260619 16:58:59.101055 30543 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:59.096407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:60933 (local address 127.20.231.190:41629)
0619 16:58:59.096524 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:59.096527 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:59.096564 (+ 37us) server_negotiation.cc:407] Connection header received
0619 16:58:59.096640 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:59.096641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:59.096653 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:59.096824 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:59.097121 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:59.097640 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:59.098381 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:59.098573 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:59.098628 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:59.100954 (+ 2326us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.129:60933: BlockingRecv error: recv got EOF from 127.20.231.129:60933 (error 108)
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":50,"threads_started":1}
Jun 19 16:58:59 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:58:59 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:58:59.242583 30544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:59.237557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.130:32831)
0619 16:58:59.237684 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:59.237696 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:59.237787 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:59.237992 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:59.237993 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:59.238171 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:59.238220 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:59.238222 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:59.239057 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:59.239058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:59.239824 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:59.239826 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:59.240178 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:59.240178 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:59.240217 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:59.240712 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:59.240716 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:59.242514 (+ 1798us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":60,"threads_started":1}
W20260619 16:58:59.242690 30543 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:59.237633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:32831 (local address 127.20.231.190:41629)
0619 16:58:59.237732 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:59.237734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:59.237748 (+ 14us) server_negotiation.cc:407] Connection header received
0619 16:58:59.237851 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:59.237852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:59.237865 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:59.237925 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:59.238294 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:59.238950 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:59.239894 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:59.240111 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:59.240172 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:59.242614 (+ 2442us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.130:32831: BlockingRecv error: recv got EOF from 127.20.231.130:32831 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Jun 19 16:58:59 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:58:59 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:58:59.380492 30545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:58:59.375561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.131:44803)
0619 16:58:59.375743 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 16:58:59.375763 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 16:58:59.375833 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:58:59.375974 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:58:59.375975 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:58:59.376149 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:58:59.376196 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:59.376198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:59.376892 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:59.376893 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:59.377440 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:58:59.377442 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:58:59.377788 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:58:59.377789 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:58:59.377846 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:59.378338 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:58:59.378342 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:58:59.380345 (+ 2003us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":112,"threads_started":1}
W20260619 16:58:59.380523 30543 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:58:59.375650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:44803 (local address 127.20.231.190:41629)
0619 16:58:59.375752 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0619 16:58:59.375754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:58:59.375819 (+ 65us) server_negotiation.cc:407] Connection header received
0619 16:58:59.375888 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:58:59.375889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:58:59.375907 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:58:59.375966 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:58:59.376308 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:59.376804 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:59.377525 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:58:59.377712 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:58:59.377789 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:58:59.380438 (+ 2649us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.131:44803: BlockingRecv error: recv got EOF from 127.20.231.131:44803 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Jun 19 16:59:00 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:00 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:00.106288 30546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:00.101434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.129:49075)
0619 16:59:00.101598 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:00.101613 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:00.101701 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:00.101853 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:00.101853 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:00.102013 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:00.102061 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:00.102062 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:00.102745 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:00.102745 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:00.103299 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:00.103300 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:00.103787 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:00.103788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:00.103837 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:00.104419 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:00.104423 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:00.106232 (+ 1809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":70,"threads_started":1}
W20260619 16:59:00.106460 30547 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:00.101543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:49075 (local address 127.20.231.190:41629)
0619 16:59:00.101696 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:00.101698 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:00.101708 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:59:00.101751 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:00.101752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:00.101765 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:00.101834 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:00.102130 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:00.102657 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:00.103395 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:00.103635 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:00.103737 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:00.106364 (+ 2627us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.129:49075: BlockingRecv error: recv got EOF from 127.20.231.129:49075 (error 108)
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":97,"threads_started":1}
Jun 19 16:59:00 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:00 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:00.247983 30548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:00.243101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.130:58403)
0619 16:59:00.243304 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:00.243318 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:00.243393 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:00.243524 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:00.243525 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:00.243692 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:00.243745 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:00.243747 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:00.244393 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:00.244394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:00.244931 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:00.244933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:00.245303 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:00.245304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:00.245346 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:00.245792 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:00.245797 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:00.247883 (+ 2086us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":141,"threads_started":1}
W20260619 16:59:00.248040 30547 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:00.243175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:58403 (local address 127.20.231.190:41629)
0619 16:59:00.243256 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:00.243258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:00.243398 (+ 140us) server_negotiation.cc:407] Connection header received
0619 16:59:00.243440 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:00.243440 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:00.243452 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:00.243506 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:00.243815 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:00.244317 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:00.245030 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:00.245232 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:00.245298 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:00.247967 (+ 2669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.130:58403: BlockingRecv error: recv got EOF from 127.20.231.130:58403 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Jun 19 16:59:00 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:00 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:00.385625 30549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:00.381009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.131:47887)
0619 16:59:00.381141 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:00.381153 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:00.381209 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:00.381351 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:00.381352 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:00.381531 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:00.381577 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:00.381579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:00.382303 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:00.382304 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:00.382835 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:00.382837 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:00.383210 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:00.383212 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:00.383284 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:00.383767 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:00.383772 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:00.385508 (+ 1736us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":75,"threads_started":1}
W20260619 16:59:00.385670 30547 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:00.381142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:47887 (local address 127.20.231.190:41629)
0619 16:59:00.381205 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:00.381207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:00.381216 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:59:00.381264 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:00.381264 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:00.381279 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:00.381345 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:00.381653 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:00.382188 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:00.382945 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:00.383130 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:00.383210 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:00.385604 (+ 2394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.131:47887: BlockingRecv error: recv got EOF from 127.20.231.131:47887 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Jun 19 16:59:01 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:01 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:01.111616 30550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:01.106746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.129:39905)
0619 16:59:01.106924 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:01.106936 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:01.106994 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:01.107117 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:01.107118 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:01.107281 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:01.107342 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:01.107344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:01.108119 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:01.108120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:01.108647 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:01.108649 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:01.109018 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:01.109019 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:01.109083 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:01.109619 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:01.109624 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:01.111556 (+ 1932us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":126,"threads_started":1}
W20260619 16:59:01.111786 30551 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:01.106884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:39905 (local address 127.20.231.190:41629)
0619 16:59:01.107002 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:01.107004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:01.107013 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:59:01.107036 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:01.107037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:01.107047 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:01.107106 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:01.107436 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:01.108012 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:01.108732 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:01.108925 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:01.108984 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:01.111675 (+ 2691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.129:39905: BlockingRecv error: recv got EOF from 127.20.231.129:39905 (error 108)
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":72,"threads_started":1}
Jun 19 16:59:01 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:01 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:01.253820 30552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:01.248519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.130:50713)
0619 16:59:01.248823 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:01.248835 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:01.248897 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:01.249018 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:01.249019 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:01.249197 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:01.249247 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:01.249249 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:01.249951 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:01.249952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:01.250680 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:01.250683 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:01.251157 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:01.251158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:01.251220 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:01.251796 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:01.251800 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:01.253714 (+ 1914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":200,"threads_started":1}
W20260619 16:59:01.253883 30551 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:01.248614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:50713 (local address 127.20.231.190:41629)
0619 16:59:01.248783 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:01.248786 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:01.248893 (+ 107us) server_negotiation.cc:407] Connection header received
0619 16:59:01.248945 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:01.248946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:01.248957 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:01.249012 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:01.249334 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:01.249866 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:01.250769 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:01.251042 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:01.251146 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:01.253817 (+ 2671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.130:50713: BlockingRecv error: recv got EOF from 127.20.231.130:50713 (error 108)
Metrics: {"server-negotiator.queue_time_us":141}
Jun 19 16:59:01 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:01 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:01.391215 30553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:01.386159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.131:48421)
0619 16:59:01.386441 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:01.386452 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:01.386532 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:01.386682 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:01.386683 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:01.386938 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:01.386987 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:01.386990 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:01.387711 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:01.387712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:01.388250 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:01.388252 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:01.388641 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:01.388642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:01.388698 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:01.389235 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:01.389241 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:01.391157 (+ 1916us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":102,"threads_started":1}
W20260619 16:59:01.391345 30551 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:01.386253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:48421 (local address 127.20.231.190:41629)
0619 16:59:01.386433 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:01.386436 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:01.386512 (+ 76us) server_negotiation.cc:407] Connection header received
0619 16:59:01.386590 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:01.386591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:01.386607 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:01.386677 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:01.387073 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:01.387632 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:01.388348 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:01.388575 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:01.388644 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:01.391276 (+ 2632us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.131:48421: BlockingRecv error: recv got EOF from 127.20.231.131:48421 (error 108)
Metrics: {"server-negotiator.queue_time_us":151}
Jun 19 16:59:02 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:02 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:02.117609 30554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:02.112130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.129:53633)
0619 16:59:02.112272 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:02.112282 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:02.112331 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:02.112540 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:02.112541 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:02.112726 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:02.112777 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:02.112787 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:02.113490 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:02.113491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:02.114177 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:02.114180 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:02.114584 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:02.114585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:02.114638 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:02.115378 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:02.115383 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:02.117510 (+ 2127us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":78,"threads_started":1}
W20260619 16:59:02.117666 30555 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:02.112238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:53633 (local address 127.20.231.190:41629)
0619 16:59:02.112373 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:02.112376 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:02.112384 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:59:02.112405 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:02.112406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:02.112416 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:02.112493 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:02.112898 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:02.113408 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:02.114285 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:02.114484 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:02.114550 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:02.117591 (+ 3041us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.129:53633: BlockingRecv error: recv got EOF from 127.20.231.129:53633 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":76,"threads_started":1}
Jun 19 16:59:02 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:02 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:02.259479 30556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:02.254355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.130:40849)
0619 16:59:02.254510 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:02.254520 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:02.254577 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:02.254704 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:02.254705 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:02.254932 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:02.254984 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:02.254986 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:02.255834 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:02.255835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:02.256359 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:02.256360 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:02.256734 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:02.256735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:02.256779 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:02.257223 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:02.257228 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:02.259378 (+ 2150us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":97,"threads_started":1}
W20260619 16:59:02.259585 30555 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:02.254438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:40849 (local address 127.20.231.190:41629)
0619 16:59:02.254508 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:02.254510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:02.254576 (+ 66us) server_negotiation.cc:407] Connection header received
0619 16:59:02.254620 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:02.254621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:02.254632 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:02.254690 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:02.255094 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:02.255749 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:02.256439 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:02.256660 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:02.256731 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:02.259492 (+ 2761us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.130:40849: BlockingRecv error: recv got EOF from 127.20.231.130:40849 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Jun 19 16:59:02 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:02 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:02.396549 30557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:02.391704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.131:42353)
0619 16:59:02.391876 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:02.391887 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:02.391948 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:02.392071 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:02.392072 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:02.392237 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:02.392289 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:02.392291 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:02.392933 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:02.392934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:02.393555 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:02.393557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:02.393939 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:02.393940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:02.393989 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:02.394519 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:02.394523 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:02.396490 (+ 1967us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":105,"threads_started":1}
W20260619 16:59:02.396658 30555 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:02.391835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:42353 (local address 127.20.231.190:41629)
0619 16:59:02.391884 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:02.391886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:02.391932 (+ 46us) server_negotiation.cc:407] Connection header received
0619 16:59:02.392001 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:02.392001 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:02.392013 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:02.392052 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:02.392356 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:02.392836 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:02.393660 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:02.393856 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:02.393910 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:02.396584 (+ 2674us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.131:42353: BlockingRecv error: recv got EOF from 127.20.231.131:42353 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Jun 19 16:59:03 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:03 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:03.122802 30558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:03.118096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.129:42093)
0619 16:59:03.118304 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:03.118314 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:03.118370 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:03.118551 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:03.118552 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:03.118721 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:03.118807 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:03.118808 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:03.119584 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:03.119585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:03.120161 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:03.120163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:03.120528 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:03.120528 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:03.120568 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:03.120995 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:03.121000 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:03.122706 (+ 1706us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":141,"threads_started":1}
W20260619 16:59:03.122927 30559 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:03.118183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:42093 (local address 127.20.231.190:41629)
0619 16:59:03.118316 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:03.118318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:03.118358 (+ 40us) server_negotiation.cc:407] Connection header received
0619 16:59:03.118430 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:03.118431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:03.118441 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:03.118492 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:03.118979 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:03.119499 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:03.120267 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:03.120460 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:03.120517 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:03.122811 (+ 2294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.129:42093: BlockingRecv error: recv got EOF from 127.20.231.129:42093 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":74,"threads_started":1}
Jun 19 16:59:03 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:03 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:03.265005 30560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:03.259930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.130:38129)
0619 16:59:03.260128 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:03.260140 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:03.260215 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:03.260384 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:03.260385 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:03.260557 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:03.260606 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:03.260608 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:03.261368 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:03.261368 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:03.261924 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:03.261925 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:03.262313 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:03.262314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:03.262365 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:03.262921 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:03.262926 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:03.264894 (+ 1968us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":149,"threads_started":1}
W20260619 16:59:03.265060 30559 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:03.260067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:38129 (local address 127.20.231.190:41629)
0619 16:59:03.260133 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:03.260136 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:03.260193 (+ 57us) server_negotiation.cc:407] Connection header received
0619 16:59:03.260300 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:03.260300 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:03.260311 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:03.260368 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:03.260776 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:03.261291 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:03.262007 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:03.262208 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:03.262270 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:03.264993 (+ 2723us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.130:38129: BlockingRecv error: recv got EOF from 127.20.231.130:38129 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Jun 19 16:59:03 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:03 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:03.407370 30561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:03.397106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.131:50737)
0619 16:59:03.397339 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:03.397351 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:03.397455 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:03.397707 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:03.397708 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:03.397892 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:03.397954 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:03.397956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:03.398731 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:03.398732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:03.399326 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:03.399328 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:03.399733 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:03.399735 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:03.399797 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:03.405296 (+ 5499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:03.405301 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:03.407274 (+ 1973us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":108,"threads_started":1}
W20260619 16:59:03.407447 30559 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:03.397236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:50737 (local address 127.20.231.190:41629)
0619 16:59:03.397312 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:03.397315 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:03.397433 (+ 118us) server_negotiation.cc:407] Connection header received
0619 16:59:03.397522 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:03.397523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:03.397541 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:03.397629 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:03.398047 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:03.398628 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:03.399423 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:03.399622 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:03.399721 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:03.407377 (+ 7656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.131:50737: BlockingRecv error: recv got EOF from 127.20.231.131:50737 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Jun 19 16:59:04 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:04 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:04.127836 30562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:04.123411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.129:52445)
0619 16:59:04.123553 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:04.123564 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:04.123618 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:04.123748 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:04.123749 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:04.123899 (+ 150us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:04.123947 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:04.123948 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:04.124597 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:04.124598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:04.125172 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:04.125174 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:04.125523 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:04.125524 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:04.125565 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:04.126029 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:04.126033 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:04.127776 (+ 1743us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":82,"threads_started":1}
W20260619 16:59:04.128129 30563 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:04.123491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:52445 (local address 127.20.231.190:41629)
0619 16:59:04.123616 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:04.123618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:04.123626 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:59:04.123664 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:04.123664 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:04.123674 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:04.123731 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:04.124016 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:04.124530 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:04.125248 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:04.125457 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:04.125510 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:04.128074 (+ 2564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.129:52445: BlockingRecv error: recv got EOF from 127.20.231.129:52445 (error 108)
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":56,"threads_started":1}
Jun 19 16:59:04 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:04 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:04.270305 30564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:04.265660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.130:44267)
0619 16:59:04.265834 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:04.265844 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:04.265903 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:04.266047 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:04.266048 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:04.266210 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:04.266268 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:04.266269 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:04.266970 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:04.266971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:04.267505 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:04.267507 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:04.267882 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:04.267884 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:04.267948 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:04.268454 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:04.268459 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:04.270224 (+ 1765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":75,"threads_started":1}
W20260619 16:59:04.270414 30563 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:04.265711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:44267 (local address 127.20.231.190:41629)
0619 16:59:04.265795 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:04.265797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:04.265903 (+ 106us) server_negotiation.cc:407] Connection header received
0619 16:59:04.265961 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:04.265962 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:04.265973 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:04.266015 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:04.266333 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:04.266834 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:04.267595 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:04.267797 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:04.267874 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:04.270346 (+ 2472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.130:44267: BlockingRecv error: recv got EOF from 127.20.231.130:44267 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Jun 19 16:59:04 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:04 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:04.412544 30565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:04.407995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.131:58675)
0619 16:59:04.408129 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:04.408146 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:04.408203 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:04.408375 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:04.408375 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:04.408532 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:04.408581 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:04.408583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:04.409290 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:04.409291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:04.409843 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:04.409845 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:04.410162 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:04.410163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:04.410204 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:04.410729 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:04.410734 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:04.412487 (+ 1753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":85,"threads_started":1}
W20260619 16:59:04.412659 30563 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:04.408045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:58675 (local address 127.20.231.190:41629)
0619 16:59:04.408105 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:04.408107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:04.408188 (+ 81us) server_negotiation.cc:407] Connection header received
0619 16:59:04.408294 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:04.408294 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:04.408305 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:04.408354 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:04.408668 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:04.409203 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:04.409919 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:04.410096 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:04.410150 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:04.412582 (+ 2432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.131:58675: BlockingRecv error: recv got EOF from 127.20.231.131:58675 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Jun 19 16:59:05 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:05 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:05.132836 30566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:05.128348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.129:39233)
0619 16:59:05.128494 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:05.128505 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:05.128551 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:05.128706 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:05.128707 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:05.128875 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:05.128923 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:05.128924 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:05.129560 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:05.129561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:05.130076 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:05.130077 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:05.130421 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:05.130422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:05.130461 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:05.130939 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:05.130943 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:05.132763 (+ 1820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":82,"threads_started":1}
W20260619 16:59:05.132969 30567 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:05.128432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:39233 (local address 127.20.231.190:41629)
0619 16:59:05.128580 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:05.128582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:05.128590 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:59:05.128611 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:05.128612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:05.128622 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:05.128694 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:05.128993 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:05.129494 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:05.130150 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:05.130356 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:05.130412 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:05.132887 (+ 2475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.129:39233: BlockingRecv error: recv got EOF from 127.20.231.129:39233 (error 108)
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":96,"threads_started":1}
Jun 19 16:59:05 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:05 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:05.275893 30568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:05.270834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.130:45117)
0619 16:59:05.271024 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:05.271037 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:05.271114 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:05.271259 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:05.271259 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:05.271443 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:05.271491 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:05.271492 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:05.272217 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:05.272258 (+ 41us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:05.272793 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:05.272796 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:05.273176 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:05.273177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:05.273241 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:05.273835 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:05.273840 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:05.275784 (+ 1944us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":117,"threads_started":1}
W20260619 16:59:05.275942 30567 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:05.270962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:45117 (local address 127.20.231.190:41629)
0619 16:59:05.271020 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:05.271022 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:05.271093 (+ 71us) server_negotiation.cc:407] Connection header received
0619 16:59:05.271175 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:05.271176 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:05.271189 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:05.271237 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:05.271571 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:05.272119 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:05.272886 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:05.273088 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:05.273174 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:05.275873 (+ 2699us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.130:45117: BlockingRecv error: recv got EOF from 127.20.231.130:45117 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Jun 19 16:59:05 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:05 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:05.417695 30569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:05.413067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.131:44969)
0619 16:59:05.413256 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:05.413267 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:05.413331 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:05.413477 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:05.413477 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:05.413639 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:05.413689 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:05.413690 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:05.414445 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:05.414446 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:05.415054 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:05.415057 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:05.415388 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:05.415389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:05.415436 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:05.415865 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:05.415869 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:05.417627 (+ 1758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":123,"threads_started":1}
W20260619 16:59:05.417847 30567 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:05.413181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:44969 (local address 127.20.231.190:41629)
0619 16:59:05.413236 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:05.413239 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:05.413319 (+ 80us) server_negotiation.cc:407] Connection header received
0619 16:59:05.413379 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:05.413380 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:05.413394 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:05.413462 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:05.413839 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:05.414369 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:05.415128 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:05.415323 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:05.415378 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:05.417714 (+ 2336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.131:44969: BlockingRecv error: recv got EOF from 127.20.231.131:44969 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Jun 19 16:59:06 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:06 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:06.138775 30570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:06.133430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.129:35555)
0619 16:59:06.133628 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:06.133643 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:06.133715 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:06.133915 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:06.133916 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:06.134084 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:06.134135 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:06.134136 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:06.135012 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:06.135013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:06.135703 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:06.135706 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:06.136064 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:06.136065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:06.136113 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:06.136622 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:06.136627 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:06.138674 (+ 2047us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":97,"threads_started":1}
W20260619 16:59:06.138926 30571 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:06.133588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:35555 (local address 127.20.231.190:41629)
0619 16:59:06.133783 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:06.133785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:06.133793 (+ 8us) server_negotiation.cc:407] Connection header received
0619 16:59:06.133816 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:06.133816 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:06.133827 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:06.133891 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:06.134239 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:06.134920 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:06.135793 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:06.135990 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:06.136049 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:06.138764 (+ 2715us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.129:35555: BlockingRecv error: recv got EOF from 127.20.231.129:35555 (error 108)
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":75,"threads_started":1}
Jun 19 16:59:06 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:06 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:06.281750 30572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:06.276446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.130:35445)
0619 16:59:06.276682 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:06.276692 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:06.276749 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:06.276908 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:06.276909 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:06.277069 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:06.277119 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:06.277120 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:06.278105 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:06.278106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:06.278651 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:06.278653 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:06.279159 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:06.279160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:06.279201 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:06.279695 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:06.279699 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:06.281649 (+ 1950us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":177,"threads_started":1}
W20260619 16:59:06.281806 30571 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:06.276519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:35445 (local address 127.20.231.190:41629)
0619 16:59:06.276591 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:06.276593 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:06.276794 (+ 201us) server_negotiation.cc:407] Connection header received
0619 16:59:06.276821 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:06.276822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:06.276832 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:06.276888 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:06.277225 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:06.278014 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:06.278776 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:06.279092 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:06.279167 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:06.281739 (+ 2572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.130:35445: BlockingRecv error: recv got EOF from 127.20.231.130:35445 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Jun 19 16:59:06 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:06 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:06.423880 30573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:06.418335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.131:51687)
0619 16:59:06.418544 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:06.418557 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:06.418632 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:06.418875 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:06.418876 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:06.419047 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:06.419096 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:06.419098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:06.420054 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:06.420056 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:06.420817 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:06.420820 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:06.421178 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:06.421179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:06.421222 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:06.421791 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:06.421795 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:06.423770 (+ 1975us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":125,"threads_started":1}
W20260619 16:59:06.423997 30571 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:06.418429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:51687 (local address 127.20.231.190:41629)
0619 16:59:06.418516 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:06.418518 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:06.418718 (+ 200us) server_negotiation.cc:407] Connection header received
0619 16:59:06.418751 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:06.418752 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:06.418767 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:06.418831 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:06.419203 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:06.419911 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:06.420900 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:06.421106 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:06.421167 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:06.423922 (+ 2755us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.131:51687: BlockingRecv error: recv got EOF from 127.20.231.131:51687 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Jun 19 16:59:07 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:07 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:07.144635 30574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:07.139434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.129:55329)
0619 16:59:07.139582 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:07.139596 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:07.139650 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:07.139846 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:07.139846 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:07.140008 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:07.140057 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:07.140058 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:07.140828 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:07.140830 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:07.141556 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:07.141558 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:07.141952 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:07.141953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:07.141998 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:07.142540 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:07.142545 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:07.144577 (+ 2032us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
W20260619 16:59:07.144870 30575 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:07.139488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:55329 (local address 127.20.231.190:41629)
0619 16:59:07.139719 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:07.139721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:07.139730 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:59:07.139752 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:07.139752 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:07.139762 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:07.139830 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:07.140132 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:07.140716 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:07.141646 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:07.141873 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:07.141942 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:07.144794 (+ 2852us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.129:55329: BlockingRecv error: recv got EOF from 127.20.231.129:55329 (error 108)
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":104,"threads_started":1}
Jun 19 16:59:07 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:07 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:07.287578 30576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:07.282330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.130:34313)
0619 16:59:07.282494 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:07.282509 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:07.282579 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:07.282754 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:07.282755 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:07.283040 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:07.283094 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:07.283097 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:07.283829 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:07.283830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:07.284397 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:07.284399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:07.284760 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:07.284761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:07.284822 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:07.285369 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:07.285374 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:07.287501 (+ 2127us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":70,"threads_started":1}
W20260619 16:59:07.287781 30575 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:07.282454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:34313 (local address 127.20.231.190:41629)
0619 16:59:07.282525 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:07.282527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:07.282563 (+ 36us) server_negotiation.cc:407] Connection header received
0619 16:59:07.282642 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:07.282643 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:07.282653 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:07.282697 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:07.283246 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:07.283739 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:07.284479 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:07.284664 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:07.284739 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:07.287645 (+ 2906us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.130:34313: BlockingRecv error: recv got EOF from 127.20.231.130:34313 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Jun 19 16:59:07 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:07 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:07.429121 30577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:07.424476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.131:36121)
0619 16:59:07.424665 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:07.424676 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:07.424743 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:07.424880 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:07.424881 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:07.425036 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:07.425085 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:07.425087 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:07.425782 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:07.425783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:07.426339 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:07.426341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:07.426665 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:07.426666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:07.426705 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:07.427218 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:07.427222 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:07.429055 (+ 1833us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":111,"threads_started":1}
W20260619 16:59:07.429257 30575 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:07.424534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:36121 (local address 127.20.231.190:41629)
0619 16:59:07.424607 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:07.424610 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:07.424744 (+ 134us) server_negotiation.cc:407] Connection header received
0619 16:59:07.424792 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:07.424793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:07.424806 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:07.424859 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:07.425190 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:07.425712 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:07.426419 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:07.426601 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:07.426656 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:07.429155 (+ 2499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.131:36121: BlockingRecv error: recv got EOF from 127.20.231.131:36121 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Jun 19 16:59:08 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:08 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:08.150593 30578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:08.145177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.129:39585)
0619 16:59:08.145393 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:08.145409 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:08.145481 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:08.145633 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:08.145634 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:08.145822 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:08.145878 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:08.145880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:08.146601 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:08.146602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:08.147180 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:08.147182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:08.147608 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:08.147609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:08.147738 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:08.148266 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:08.148272 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:08.150480 (+ 2208us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":105,"threads_started":1}
W20260619 16:59:08.150661 30579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:08.145277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:39585 (local address 127.20.231.190:41629)
0619 16:59:08.145458 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:08.145460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:08.145470 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:59:08.145544 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:08.145545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:08.145558 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:08.145610 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:08.145965 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:08.146505 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:08.147284 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:08.147482 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:08.147542 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:08.150586 (+ 3044us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.129:39585: BlockingRecv error: recv got EOF from 127.20.231.129:39585 (error 108)
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":84,"threads_started":1}
Jun 19 16:59:08 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:08 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:08.293783 30580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:08.288261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.130:54875)
0619 16:59:08.288422 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:08.288433 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:08.288544 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:08.288710 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:08.288711 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:08.288910 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:08.288959 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:08.288962 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:08.289700 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:08.289701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:08.290583 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:08.290585 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:08.291222 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:08.291223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:08.291295 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:08.291790 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:08.291795 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:08.293718 (+ 1923us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":78,"threads_started":1}
W20260619 16:59:08.293959 30579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:08.288323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:54875 (local address 127.20.231.190:41629)
0619 16:59:08.288375 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:08.288377 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:08.288525 (+ 148us) server_negotiation.cc:407] Connection header received
0619 16:59:08.288616 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:08.288617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:08.288637 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:08.288703 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:08.289048 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:08.289607 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:08.290718 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:08.291100 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:08.291202 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:08.293888 (+ 2686us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.130:54875: BlockingRecv error: recv got EOF from 127.20.231.130:54875 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Jun 19 16:59:08 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:08 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:08.439685 30581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:08.429751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.131:40929)
0619 16:59:08.429918 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:08.429931 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:08.430025 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:08.430290 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:08.430291 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:08.430489 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:08.430548 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:08.430553 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:08.431561 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:08.431563 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:08.432168 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:08.432170 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:08.432626 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:08.432628 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:08.432701 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:08.437601 (+ 4900us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:08.437607 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:08.439625 (+ 2018us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":98,"threads_started":1}
W20260619 16:59:08.439787 30579 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:08.429917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:40929 (local address 127.20.231.190:41629)
0619 16:59:08.429998 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:08.430001 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:08.430013 (+ 12us) server_negotiation.cc:407] Connection header received
0619 16:59:08.430156 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:08.430158 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:08.430177 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:08.430257 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:08.430670 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:08.431439 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:08.432290 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:08.432512 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:08.432615 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:08.439731 (+ 7116us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.131:40929: BlockingRecv error: recv got EOF from 127.20.231.131:40929 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Jun 19 16:59:09 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:09 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:09.156550 30582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:09.151288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.129:50311)
0619 16:59:09.151452 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:09.151462 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:09.151515 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:09.151675 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:09.151676 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:09.151900 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:09.151946 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:09.151949 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:09.152748 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:09.152748 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:09.153327 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:09.153329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:09.153910 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:09.153911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:09.153979 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:09.154538 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:09.154543 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:09.156469 (+ 1926us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":106,"threads_started":1}
W20260619 16:59:09.156798 30583 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:09.151357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:50311 (local address 127.20.231.190:41629)
0619 16:59:09.151525 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:09.151527 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:09.151538 (+ 11us) server_negotiation.cc:407] Connection header received
0619 16:59:09.151564 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:09.151565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:09.151577 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:09.151660 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:09.152044 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:09.152587 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:09.153391 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:09.153814 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:09.153878 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:09.156720 (+ 2842us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.129:50311: BlockingRecv error: recv got EOF from 127.20.231.129:50311 (error 108)
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":97,"threads_started":1}
Jun 19 16:59:09 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:09 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:09.299618 30584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:09.294437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.130:35333)
0619 16:59:09.294601 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:09.294612 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:09.294669 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:09.294929 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:09.294931 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:09.295161 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:09.295206 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:09.295208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:09.295903 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:09.295904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:09.296631 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:09.296635 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:09.297077 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:09.297079 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:09.297138 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:09.297662 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:09.297668 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:09.299507 (+ 1839us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":107,"threads_started":1}
W20260619 16:59:09.299665 30583 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:09.294519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:35333 (local address 127.20.231.190:41629)
0619 16:59:09.294605 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:09.294607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:09.294673 (+ 66us) server_negotiation.cc:407] Connection header received
0619 16:59:09.294747 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:09.294748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:09.294760 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:09.294808 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:09.295282 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:09.295826 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:09.296728 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:09.296994 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:09.297077 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:09.299598 (+ 2521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.130:35333: BlockingRecv error: recv got EOF from 127.20.231.130:35333 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Jun 19 16:59:09 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:09 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:09.445796 30585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:09.440327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.131:58539)
0619 16:59:09.440514 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:09.440529 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:09.440646 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:09.440862 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:09.440863 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:09.441061 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:09.441140 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:09.441143 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:09.441970 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:09.441971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:09.442580 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:09.442582 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:09.443066 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:09.443067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:09.443113 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:09.443674 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:09.443678 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:09.445717 (+ 2039us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":92,"threads_started":1}
W20260619 16:59:09.445977 30583 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:09.440483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:58539 (local address 127.20.231.190:41629)
0619 16:59:09.440566 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:09.440575 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:09.440625 (+ 50us) server_negotiation.cc:407] Connection header received
0619 16:59:09.440739 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:09.440740 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:09.440758 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:09.440834 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:09.441251 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:09.441849 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:09.442684 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:09.442950 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:09.443063 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:09.445879 (+ 2816us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.131:58539: BlockingRecv error: recv got EOF from 127.20.231.131:58539 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Jun 19 16:59:10 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:10 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:10.162426 30586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:10.157093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.129:33817)
0619 16:59:10.157275 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:10.157291 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:10.157369 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:10.157519 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:10.157519 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:10.157714 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:10.157828 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:10.157830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:10.158467 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:10.158468 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:10.159301 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:10.159304 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:10.159690 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:10.159691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:10.159775 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:10.160359 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:10.160365 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:10.162350 (+ 1985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":72,"threads_started":1}
W20260619 16:59:10.162539 30587 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:10.157160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:33817 (local address 127.20.231.190:41629)
0619 16:59:10.157348 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:10.157349 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:10.157359 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:59:10.157423 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:10.157424 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:10.157434 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:10.157495 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:10.157891 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:10.158388 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:10.159387 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:10.159609 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:10.159666 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:10.162458 (+ 2792us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.129:33817: BlockingRecv error: recv got EOF from 127.20.231.129:33817 (error 108)
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":123,"threads_started":1}
Jun 19 16:59:10 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:10 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:10.305548 30588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:10.300109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.130:50401)
0619 16:59:10.300355 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:10.300369 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:10.300466 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:10.300741 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:10.300742 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:10.300924 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:10.300974 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:10.300975 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:10.301718 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:10.301719 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:10.302317 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:10.302319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:10.302722 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:10.302723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:10.302792 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:10.303334 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:10.303339 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:10.305488 (+ 2149us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":124,"threads_started":1}
W20260619 16:59:10.305733 30587 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:10.300229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:50401 (local address 127.20.231.190:41629)
0619 16:59:10.300341 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:10.300344 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:10.300447 (+ 103us) server_negotiation.cc:407] Connection header received
0619 16:59:10.300560 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:10.300561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:10.300575 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:10.300652 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:10.301067 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:10.301624 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:10.302411 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:10.302628 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:10.302713 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:10.305662 (+ 2949us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.130:50401: BlockingRecv error: recv got EOF from 127.20.231.130:50401 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Jun 19 16:59:10 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:10 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:10.451849 30589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:10.446370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.131:52741)
0619 16:59:10.446559 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:10.446578 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:10.446711 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:10.446972 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:10.446973 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:10.447165 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:10.447209 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:10.447211 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:10.447916 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:10.447917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:10.448438 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:10.448439 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:10.449021 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:10.449022 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:10.449070 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:10.449617 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:10.449621 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:10.451707 (+ 2086us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":134,"threads_started":1}
W20260619 16:59:10.451849 30587 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:10.446451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:52741 (local address 127.20.231.190:41629)
0619 16:59:10.446522 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:10.446524 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:10.446640 (+ 116us) server_negotiation.cc:407] Connection header received
0619 16:59:10.446820 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:10.446822 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:10.446835 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:10.446956 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:10.447283 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:10.447833 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:10.448539 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:10.448899 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:10.449014 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:10.451795 (+ 2781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.131:52741: BlockingRecv error: recv got EOF from 127.20.231.131:52741 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Jun 19 16:59:11 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:11 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:11.168458 30590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:11.163061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.129:35401)
0619 16:59:11.163213 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:11.163225 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:11.163294 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:11.163452 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:11.163453 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:11.163618 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:11.163665 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:11.163668 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:11.164441 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:11.164442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:11.165124 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:11.165127 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:11.165544 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:11.165545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:11.165606 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:11.166091 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:11.166096 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:11.168351 (+ 2255us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":89,"threads_started":1}
W20260619 16:59:11.168532 30591 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:11.163131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:35401 (local address 127.20.231.190:41629)
0619 16:59:11.163300 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:11.163302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:11.163315 (+ 13us) server_negotiation.cc:407] Connection header received
0619 16:59:11.163342 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:11.163343 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:11.163354 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:11.163434 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:11.163760 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:11.164350 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:11.165214 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:11.165454 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:11.165535 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:11.168452 (+ 2917us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.129:35401: BlockingRecv error: recv got EOF from 127.20.231.129:35401 (error 108)
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":101,"threads_started":1}
Jun 19 16:59:11 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:11 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:11.311782 30592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:11.306271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.130:47143)
0619 16:59:11.306546 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:11.306561 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:11.306644 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:11.306936 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:11.306937 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:11.307254 (+ 317us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:11.307303 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:11.307306 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:11.308090 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:11.308091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:11.308786 (+ 695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:11.308788 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:11.309220 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:11.309221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:11.309282 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:11.309845 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:11.309849 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:11.311712 (+ 1863us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":75,"threads_started":1}
W20260619 16:59:11.311942 30591 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:11.306531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:47143 (local address 127.20.231.190:41629)
0619 16:59:11.306596 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:11.306598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:11.306626 (+ 28us) server_negotiation.cc:407] Connection header received
0619 16:59:11.306708 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:11.306709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:11.306720 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:11.306824 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:11.307417 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:11.307995 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:11.308870 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:11.309124 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:11.309210 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:11.311843 (+ 2633us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.130:47143: BlockingRecv error: recv got EOF from 127.20.231.130:47143 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Jun 19 16:59:11 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:11 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:11.457067 30593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:11.452384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.131:57173)
0619 16:59:11.452507 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:11.452519 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:11.452595 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:11.452792 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:11.452793 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:11.452973 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:11.453017 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:11.453019 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:11.453690 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:11.453691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:11.454255 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:11.454257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:11.454625 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:11.454626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:11.454698 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:11.455168 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:11.455172 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:11.456955 (+ 1783us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":67,"threads_started":1}
W20260619 16:59:11.457098 30591 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:11.452450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:57173 (local address 127.20.231.190:41629)
0619 16:59:11.452504 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:11.452506 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:11.452576 (+ 70us) server_negotiation.cc:407] Connection header received
0619 16:59:11.452707 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:11.452708 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:11.452724 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:11.452779 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:11.453112 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:11.453607 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:11.454350 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:11.454542 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:11.454624 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:11.457031 (+ 2407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.131:57173: BlockingRecv error: recv got EOF from 127.20.231.131:57173 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Jun 19 16:59:12 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:12 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:12.174639 30594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:12.169204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.129:57447)
0619 16:59:12.169373 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:12.169384 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:12.169438 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:12.169666 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:12.169666 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:12.169836 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:12.169914 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:12.169916 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:12.170820 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:12.170822 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:12.171737 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:12.171740 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:12.172135 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:12.172136 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:12.172198 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:12.172663 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:12.172668 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:12.174541 (+ 1873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":93,"threads_started":1}
W20260619 16:59:12.174711 30595 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:12.169320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:57447 (local address 127.20.231.190:41629)
0619 16:59:12.169518 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:12.169521 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:12.169530 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:59:12.169555 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:12.169556 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:12.169567 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:12.169642 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:12.170019 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:12.170595 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:12.171830 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:12.172049 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:12.172134 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:12.174627 (+ 2493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.129:57447: BlockingRecv error: recv got EOF from 127.20.231.129:57447 (error 108)
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":88,"threads_started":1}
Jun 19 16:59:12 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:12 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:12.317605 30596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:12.312313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.130:56263)
0619 16:59:12.312571 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:12.312582 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:12.312641 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:12.312785 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:12.312786 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:12.312941 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:12.312988 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:12.312990 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:12.314048 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:12.314049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:12.314596 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:12.314598 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:12.315066 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:12.315067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:12.315132 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:12.315593 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:12.315598 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:12.317540 (+ 1942us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":195,"threads_started":1}
W20260619 16:59:12.317775 30595 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:12.312403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:56263 (local address 127.20.231.190:41629)
0619 16:59:12.312469 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:12.312475 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:12.312644 (+ 169us) server_negotiation.cc:407] Connection header received
0619 16:59:12.312705 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:12.312706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:12.312718 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:12.312763 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:12.313080 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:12.313967 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:12.314675 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:12.314972 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:12.315037 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:12.317675 (+ 2638us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.130:56263: BlockingRecv error: recv got EOF from 127.20.231.130:56263 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Jun 19 16:59:12 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:12 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.131@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:12.462152 30597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:12.457574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.131:56767)
0619 16:59:12.457710 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:12.457720 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:12.457787 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:12.457916 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:12.457916 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:12.458112 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:12.458189 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:12.458191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:12.458970 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:12.458971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:12.459495 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:12.459497 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:12.459862 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:12.459864 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:12.459924 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:12.460344 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:12.460348 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:12.462093 (+ 1745us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":58,"threads_started":1}
W20260619 16:59:12.462286 30595 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:12.457609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:56767 (local address 127.20.231.190:41629)
0619 16:59:12.457678 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:12.457680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:12.457771 (+ 91us) server_negotiation.cc:407] Connection header received
0619 16:59:12.457843 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:12.457844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:12.457856 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:12.457900 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:12.458246 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:12.458888 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:12.459559 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:12.459737 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:12.459792 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:12.462203 (+ 2411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.131:56767: BlockingRecv error: recv got EOF from 127.20.231.131:56767 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
Jun 19 16:59:13 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:13 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.129@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:13.180259 30598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:13.175324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.129:33891)
0619 16:59:13.175444 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:13.175455 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:13.175522 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:13.175709 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:13.175710 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:13.175879 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:13.175931 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:13.175933 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:13.176604 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:13.176605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:13.177163 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:13.177164 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:13.177515 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:13.177516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:13.177559 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:13.178149 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:13.178154 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:13.180187 (+ 2033us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":69,"threads_started":1}
W20260619 16:59:13.180429 30599 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:13.175415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:33891 (local address 127.20.231.190:41629)
0619 16:59:13.175573 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:13.175575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:13.175585 (+ 10us) server_negotiation.cc:407] Connection header received
0619 16:59:13.175610 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:13.175611 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:13.175621 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:13.175691 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:13.176009 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:13.176529 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:13.177256 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:13.177444 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:13.177498 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:13.180349 (+ 2851us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.129:33891: BlockingRecv error: recv got EOF from 127.20.231.129:33891 (error 108)
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":97,"threads_started":1}
Jun 19 16:59:13 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
Jun 19 16:59:13 dist-test-slave-vnbj krb5kdc[29541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.20.231.130@KRBTEST.COM for oryx/127.20.231.190@KRBTEST.COM, Server not found in Kerberos database
I20260619 16:59:13.323441 30600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:13.318263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41629 (local address 127.20.231.130:34629)
0619 16:59:13.318410 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:13.318421 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:13.318476 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:13.318630 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:13.318631 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:13.318793 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:13.318865 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:13.318867 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:13.319612 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:13.319613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:13.320151 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:13.320153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:13.320542 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:13.320543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:13.320585 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:13.321146 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:13.321151 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:13.323366 (+ 2215us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.20.231.190:41629: Server oryx/127.20.231.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":83,"threads_started":1}
W20260619 16:59:13.323563 30599 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:13.318341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:34629 (local address 127.20.231.190:41629)
0619 16:59:13.318436 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:13.318438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:13.318466 (+ 28us) server_negotiation.cc:407] Connection header received
0619 16:59:13.318527 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:13.318528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:13.318539 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:13.318605 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:13.318949 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:13.319523 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:13.320266 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:13.320466 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:13.320532 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:13.323476 (+ 2944us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.20.231.130:34629: BlockingRecv error: recv got EOF from 127.20.231.130:34629 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 16:59:13.365105 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 30124
I20260619 16:59:13.369513 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 30276
I20260619 16:59:13.373700 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 30408
I20260619 16:59:13.377625 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 30054
2026-06-19T16:59:13Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17482 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-06-19T16:59:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:59:13Z Disabled control of system clock
I20260619 16:59:13.405535 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:36163
--webserver_interface=127.20.231.190
--webserver_port=0
--builtin_ntp_servers=127.20.231.148:43193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:36163
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 16:59:13.486807 30604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:59:13.487036 30604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:59:13.487056 30604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:59:13.488256 30604 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:59:13.488300 30604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:59:13.488313 30604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:59:13.488325 30604 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:59:13.488368 30604 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:59:13.489837 30604 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:43193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:36163
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:36163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:59:13.490099 30604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:59:13.490408 30604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:59:13.495623 30612 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:59:13.495664 30604 server_base.cc:1034] running on GCE node
W20260619 16:59:13.495630 30609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:59:13.495710 30610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:59:13.496112 30604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:59:13.496383 30604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:59:13.497550 30604 hybrid_clock.cc:648] HybridClock initialized: now 1781888353497535 us; error 45 us; skew 500 ppm
I20260619 16:59:13.505599 30604 webserver.cc:460] Webserver started at http://127.20.231.190:34157/ using document root <none> and password file <none>
I20260619 16:59:13.505954 30604 fs_manager.cc:359] Metadata directory not provided
I20260619 16:59:13.506038 30604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:59:13.506196 30604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:59:13.507438 30604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "cd64be1a26cf4332a30a3fe4bb3850f8"
format_stamp: "Formatted at 2026-06-19 16:59:13 on dist-test-slave-vnbj"
server_key: "67aaae0c59bc08ea7739d8c4b688cee6"
server_key_iv: "de1d6f9a2351c09f004965084c61439e"
server_key_version: "encryptionkey@0"
I20260619 16:59:13.507822 30604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "cd64be1a26cf4332a30a3fe4bb3850f8"
format_stamp: "Formatted at 2026-06-19 16:59:13 on dist-test-slave-vnbj"
server_key: "67aaae0c59bc08ea7739d8c4b688cee6"
server_key_iv: "de1d6f9a2351c09f004965084c61439e"
server_key_version: "encryptionkey@0"
I20260619 16:59:13.509181 30604 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:59:13.509961 30618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:59:13.510138 30604 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:59:13.510252 30604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "cd64be1a26cf4332a30a3fe4bb3850f8"
format_stamp: "Formatted at 2026-06-19 16:59:13 on dist-test-slave-vnbj"
server_key: "67aaae0c59bc08ea7739d8c4b688cee6"
server_key_iv: "de1d6f9a2351c09f004965084c61439e"
server_key_version: "encryptionkey@0"
I20260619 16:59:13.510329 30604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:59:13.521632 30604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:59:13.521988 30604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:59:13.522158 30604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:59:13.526371 30604 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:36163
I20260619 16:59:13.526400 30670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:36163 every 8 connection(s)
I20260619 16:59:13.526765 30604 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 16:59:13.527449 30671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:59:13.530001 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 30604
I20260619 16:59:13.530117 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 16:59:13.530156 30671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd64be1a26cf4332a30a3fe4bb3850f8: Bootstrap starting.
I20260619 16:59:13.530332 21406 external_mini_cluster.cc:1428] Setting key 4d808426739622c05d13f2ee9ca2e4cc
I20260619 16:59:13.530942 30671 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd64be1a26cf4332a30a3fe4bb3850f8: Neither blocks nor log segments found. Creating new log.
I20260619 16:59:13.531288 30671 log.cc:824] T 00000000000000000000000000000000 P cd64be1a26cf4332a30a3fe4bb3850f8: Log is configured to *not* fsync() on all Append() calls
I20260619 16:59:13.532020 30671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd64be1a26cf4332a30a3fe4bb3850f8: No bootstrap required, opened a new log
I20260619 16:59:13.533531 30671 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd64be1a26cf4332a30a3fe4bb3850f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd64be1a26cf4332a30a3fe4bb3850f8" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36163 } }
I20260619 16:59:13.533706 30671 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd64be1a26cf4332a30a3fe4bb3850f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:59:13.533741 30671 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd64be1a26cf4332a30a3fe4bb3850f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd64be1a26cf4332a30a3fe4bb3850f8, State: Initialized, Role: FOLLOWER
I20260619 16:59:13.533860 30671 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd64be1a26cf4332a30a3fe4bb3850f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd64be1a26cf4332a30a3fe4bb3850f8" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36163 } }
I20260619 16:59:13.533941 30671 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd64be1a26cf4332a30a3fe4bb3850f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:59:13.533968 30671 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd64be1a26cf4332a30a3fe4bb3850f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:59:13.534013 30671 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd64be1a26cf4332a30a3fe4bb3850f8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:59:13.535058 30671 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd64be1a26cf4332a30a3fe4bb3850f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd64be1a26cf4332a30a3fe4bb3850f8" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36163 } }
I20260619 16:59:13.535182 30671 leader_election.cc:302] T 00000000000000000000000000000000 P cd64be1a26cf4332a30a3fe4bb3850f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd64be1a26cf4332a30a3fe4bb3850f8; no voters:
I20260619 16:59:13.535362 30671 leader_election.cc:288] T 00000000000000000000000000000000 P cd64be1a26cf4332a30a3fe4bb3850f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:59:13.535537 30677 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd64be1a26cf4332a30a3fe4bb3850f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:59:13.535660 30671 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd64be1a26cf4332a30a3fe4bb3850f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:59:13.535717 30677 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd64be1a26cf4332a30a3fe4bb3850f8 [term 1 LEADER]: Becoming Leader. State: Replica: cd64be1a26cf4332a30a3fe4bb3850f8, State: Running, Role: LEADER
I20260619 16:59:13.535810 30677 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd64be1a26cf4332a30a3fe4bb3850f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd64be1a26cf4332a30a3fe4bb3850f8" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36163 } }
I20260619 16:59:13.536147 30677 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd64be1a26cf4332a30a3fe4bb3850f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd64be1a26cf4332a30a3fe4bb3850f8. Latest consensus state: current_term: 1 leader_uuid: "cd64be1a26cf4332a30a3fe4bb3850f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd64be1a26cf4332a30a3fe4bb3850f8" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36163 } } }
I20260619 16:59:13.536428 30676 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd64be1a26cf4332a30a3fe4bb3850f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd64be1a26cf4332a30a3fe4bb3850f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd64be1a26cf4332a30a3fe4bb3850f8" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36163 } } }
I20260619 16:59:13.536520 30677 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd64be1a26cf4332a30a3fe4bb3850f8 [sys.catalog]: This master's current role is: LEADER
I20260619 16:59:13.536602 30676 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd64be1a26cf4332a30a3fe4bb3850f8 [sys.catalog]: This master's current role is: LEADER
I20260619 16:59:13.536617 30674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:13.531142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33900 (local address 127.20.231.190:36163)
0619 16:59:13.531350 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:13.531358 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:13.531383 (+ 25us) server_negotiation.cc:407] Connection header received
0619 16:59:13.531846 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:13.531854 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:13.531913 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:13.532030 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:13.532330 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:13.533204 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:13.534318 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:13.534727 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:13.534832 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:13.535699 (+ 867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:59:13.535703 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:59:13.535704 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:59:13.535938 (+ 234us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:59:13.536000 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:59:13.536016 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:59:13.536074 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 16:59:13.536265 (+ 191us) server_negotiation.cc:299] Negotiation successful
0619 16:59:13.536366 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":82,"threads_started":1}
I20260619 16:59:13.536999 30686 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:59:13.537662 30686 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:59:13.539224 30686 catalog_manager.cc:1347] Generated new cluster ID: 35ca0c20d7d8415e9f7f467f1e8914d0
I20260619 16:59:13.539301 30686 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:59:13.555197 30686 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:59:13.555734 30686 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:59:13.560547 30686 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd64be1a26cf4332a30a3fe4bb3850f8: Generated new TSK 0
I20260619 16:59:13.560755 30686 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:59:13.567692 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.129:0
--local_ip_for_outbound_sockets=127.20.231.129
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:36163
--builtin_ntp_servers=127.20.231.148:43193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260619 16:59:13.660216 30695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:59:13.660466 30695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:59:13.660504 30695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:59:13.661680 30695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:59:13.661749 30695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:59:13.661800 30695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.129
I20260619 16:59:13.663231 30695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:43193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:36163
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30695
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.129
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:59:13.663494 30695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:59:13.663806 30695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:59:13.668932 30700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:59:13.669050 30703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:59:13.668917 30701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:59:13.669305 30695 server_base.cc:1034] running on GCE node
I20260619 16:59:13.669582 30695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:59:13.669870 30695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:59:13.671108 30695 hybrid_clock.cc:648] HybridClock initialized: now 1781888353671087 us; error 27 us; skew 500 ppm
I20260619 16:59:13.672749 30695 webserver.cc:460] Webserver started at http://127.20.231.129:33167/ using document root <none> and password file <none>
I20260619 16:59:13.673036 30695 fs_manager.cc:359] Metadata directory not provided
I20260619 16:59:13.673094 30695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:59:13.673233 30695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:59:13.674698 30695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/data/instance:
uuid: "5c811b46f5a6425bb78967789f752359"
format_stamp: "Formatted at 2026-06-19 16:59:13 on dist-test-slave-vnbj"
server_key: "8e537857175b153387608436c88933de"
server_key_iv: "767bfc5ead087e81e0bd2937ccb6e9c1"
server_key_version: "encryptionkey@0"
I20260619 16:59:13.675127 30695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance:
uuid: "5c811b46f5a6425bb78967789f752359"
format_stamp: "Formatted at 2026-06-19 16:59:13 on dist-test-slave-vnbj"
server_key: "8e537857175b153387608436c88933de"
server_key_iv: "767bfc5ead087e81e0bd2937ccb6e9c1"
server_key_version: "encryptionkey@0"
I20260619 16:59:13.676738 30695 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 16:59:13.677561 30709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:59:13.677788 30695 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:59:13.677882 30695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/wal
uuid: "5c811b46f5a6425bb78967789f752359"
format_stamp: "Formatted at 2026-06-19 16:59:13 on dist-test-slave-vnbj"
server_key: "8e537857175b153387608436c88933de"
server_key_iv: "767bfc5ead087e81e0bd2937ccb6e9c1"
server_key_version: "encryptionkey@0"
I20260619 16:59:13.677961 30695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:59:13.687206 30695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:59:13.687553 30695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:59:13.687696 30695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:59:13.687938 30695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:59:13.688294 30695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:59:13.688328 30695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:59:13.688349 30695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:59:13.688361 30695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:59:13.693931 30695 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.129:39375
I20260619 16:59:13.694022 30822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.129:39375 every 8 connection(s)
I20260619 16:59:13.694339 30695 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
I20260619 16:59:13.699477 30674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:13.695142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:56177 (local address 127.20.231.190:36163)
0619 16:59:13.695218 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:13.695221 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:13.695452 (+ 231us) server_negotiation.cc:407] Connection header received
0619 16:59:13.695937 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:13.695938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:13.695948 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:13.696003 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:13.696615 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:13.697114 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:13.698098 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:13.698314 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:13.698371 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:13.699212 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:59:13.699213 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:59:13.699214 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:59:13.699218 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:59:13.699261 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:59:13.699265 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:59:13.699312 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 16:59:13.699395 (+ 83us) server_negotiation.cc:299] Negotiation successful
0619 16:59:13.699416 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 16:59:13.699668 30825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:13.695119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36163 (local address 127.20.231.129:56177)
0619 16:59:13.695331 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:13.695365 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:13.695848 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:13.696055 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:13.696058 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:13.696065 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:13.696548 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:13.696551 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:13.697195 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:13.697196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:13.698024 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:13.698026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:13.698394 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:13.698395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:13.698505 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:13.699079 (+ 574us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 16:59:13.699080 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:13.699108 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 16:59:13.699131 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:59:13.699323 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:59:13.699327 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:59:13.699329 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 16:59:13.699382 (+ 53us) client_negotiation.cc:240] Negotiation successful
0619 16:59:13.699462 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":106,"threads_started":1}
I20260619 16:59:13.700204 30823 heartbeater.cc:347] Connected to a master server at 127.20.231.190:36163
I20260619 16:59:13.700327 30823 heartbeater.cc:464] Registering TS with master...
I20260619 16:59:13.700474 30823 heartbeater.cc:511] Master 127.20.231.190:36163 requested a full tablet report, sending...
I20260619 16:59:13.701012 30635 ts_manager.cc:194] Registered new tserver with Master: 5c811b46f5a6425bb78967789f752359 (127.20.231.129:39375)
I20260619 16:59:13.701640 30635 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.231.129:56177
I20260619 16:59:13.703363 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 30695
I20260619 16:59:13.703434 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance
I20260619 16:59:13.703646 21406 external_mini_cluster.cc:1428] Setting key a479527d3d713f19ad4aae1ce2a319f4
I20260619 16:59:13.704504 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.130:0
--local_ip_for_outbound_sockets=127.20.231.130
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:36163
--builtin_ntp_servers=127.20.231.148:43193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260619 16:59:13.785372 30826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:59:13.785591 30826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:59:13.785614 30826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:59:13.786823 30826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:59:13.786899 30826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:59:13.786927 30826 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.130
I20260619 16:59:13.788312 30826 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:43193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:36163
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30826
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.130
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:59:13.788586 30826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:59:13.788913 30826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:59:13.794391 30831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:59:13.794360 30832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:59:13.794366 30834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:59:13.794401 30826 server_base.cc:1034] running on GCE node
I20260619 16:59:13.794983 30826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:59:13.795245 30826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:59:13.796423 30826 hybrid_clock.cc:648] HybridClock initialized: now 1781888353796404 us; error 37 us; skew 500 ppm
I20260619 16:59:13.797894 30826 webserver.cc:460] Webserver started at http://127.20.231.130:45119/ using document root <none> and password file <none>
I20260619 16:59:13.798171 30826 fs_manager.cc:359] Metadata directory not provided
I20260619 16:59:13.798246 30826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:59:13.798403 30826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:59:13.799590 30826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/data/instance:
uuid: "24a6e0b4509649659120f9b04b21f000"
format_stamp: "Formatted at 2026-06-19 16:59:13 on dist-test-slave-vnbj"
server_key: "aaa2ab20d25eae473016b611f5d8316b"
server_key_iv: "9ea320e29c510ba4ce2b571addc8e028"
server_key_version: "encryptionkey@0"
I20260619 16:59:13.799971 30826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance:
uuid: "24a6e0b4509649659120f9b04b21f000"
format_stamp: "Formatted at 2026-06-19 16:59:13 on dist-test-slave-vnbj"
server_key: "aaa2ab20d25eae473016b611f5d8316b"
server_key_iv: "9ea320e29c510ba4ce2b571addc8e028"
server_key_version: "encryptionkey@0"
I20260619 16:59:13.801268 30826 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 16:59:13.802166 30840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:59:13.802403 30826 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:59:13.802511 30826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/wal
uuid: "24a6e0b4509649659120f9b04b21f000"
format_stamp: "Formatted at 2026-06-19 16:59:13 on dist-test-slave-vnbj"
server_key: "aaa2ab20d25eae473016b611f5d8316b"
server_key_iv: "9ea320e29c510ba4ce2b571addc8e028"
server_key_version: "encryptionkey@0"
I20260619 16:59:13.802587 30826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:59:13.814239 30826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:59:13.814561 30826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:59:13.814709 30826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:59:13.814998 30826 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:59:13.815343 30826 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:59:13.815402 30826 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:59:13.815443 30826 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:59:13.815471 30826 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:59:13.821432 30826 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.130:44431
I20260619 16:59:13.821472 30953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.130:44431 every 8 connection(s)
I20260619 16:59:13.821830 30826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
I20260619 16:59:13.827368 30674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:13.822548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:54017 (local address 127.20.231.190:36163)
0619 16:59:13.822612 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:13.822614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:13.822924 (+ 310us) server_negotiation.cc:407] Connection header received
0619 16:59:13.823464 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:13.823465 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:13.823477 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:13.823544 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:13.824284 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:13.824775 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:13.825807 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:13.825993 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:13.826052 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:13.827124 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:59:13.827125 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:59:13.827125 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:59:13.827130 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:59:13.827144 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:59:13.827148 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:59:13.827197 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 16:59:13.827283 (+ 86us) server_negotiation.cc:299] Negotiation successful
0619 16:59:13.827305 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 16:59:13.827584 30956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:13.822537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36163 (local address 127.20.231.130:54017)
0619 16:59:13.822731 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:13.822801 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:13.823347 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:13.823628 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:13.823632 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:13.823641 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:13.824178 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:13.824182 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:13.824860 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:13.824861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:13.825666 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:13.825668 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:13.826072 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:13.826072 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:13.826182 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:13.826972 (+ 790us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 16:59:13.826973 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:13.827002 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 16:59:13.827026 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:59:13.827210 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:59:13.827213 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:59:13.827216 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:59:13.827274 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 16:59:13.827360 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":100,"threads_started":1}
I20260619 16:59:13.828197 30954 heartbeater.cc:347] Connected to a master server at 127.20.231.190:36163
I20260619 16:59:13.828308 30954 heartbeater.cc:464] Registering TS with master...
I20260619 16:59:13.828500 30954 heartbeater.cc:511] Master 127.20.231.190:36163 requested a full tablet report, sending...
I20260619 16:59:13.828928 30635 ts_manager.cc:194] Registered new tserver with Master: 24a6e0b4509649659120f9b04b21f000 (127.20.231.130:44431)
I20260619 16:59:13.829378 30635 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.231.130:54017
I20260619 16:59:13.829470 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 30826
I20260619 16:59:13.829566 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance
I20260619 16:59:13.829778 21406 external_mini_cluster.cc:1428] Setting key 8088810af874846d1a3c9c3bdff21b41
I20260619 16:59:13.830672 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.131:0
--local_ip_for_outbound_sockets=127.20.231.131
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:36163
--builtin_ntp_servers=127.20.231.148:43193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260619 16:59:13.913074 30957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:59:13.913286 30957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:59:13.913309 30957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:59:13.914523 30957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:59:13.914568 30957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:59:13.914593 30957 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.131
I20260619 16:59:13.915997 30957 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:43193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:36163
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.131
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:59:13.916198 30957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:59:13.916452 30957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:59:13.921613 30963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:59:13.921574 30962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:59:13.921618 30957 server_base.cc:1034] running on GCE node
W20260619 16:59:13.921572 30965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:59:13.921967 30957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:59:13.922226 30957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:59:13.923380 30957 hybrid_clock.cc:648] HybridClock initialized: now 1781888353923372 us; error 37 us; skew 500 ppm
I20260619 16:59:13.924793 30957 webserver.cc:460] Webserver started at http://127.20.231.131:39573/ using document root <none> and password file <none>
I20260619 16:59:13.925083 30957 fs_manager.cc:359] Metadata directory not provided
I20260619 16:59:13.925158 30957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:59:13.925318 30957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:59:13.926440 30957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/data/instance:
uuid: "20654be425564e368576d79ba2584717"
format_stamp: "Formatted at 2026-06-19 16:59:13 on dist-test-slave-vnbj"
server_key: "66d4fffd74bba34a27788c403894efbd"
server_key_iv: "f7e9e6fe9f4f2299d3b0364ecb02f4e1"
server_key_version: "encryptionkey@0"
I20260619 16:59:13.926821 30957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance:
uuid: "20654be425564e368576d79ba2584717"
format_stamp: "Formatted at 2026-06-19 16:59:13 on dist-test-slave-vnbj"
server_key: "66d4fffd74bba34a27788c403894efbd"
server_key_iv: "f7e9e6fe9f4f2299d3b0364ecb02f4e1"
server_key_version: "encryptionkey@0"
I20260619 16:59:13.928195 30957 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:59:13.929008 30971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:59:13.929177 30957 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:59:13.929284 30957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/wal
uuid: "20654be425564e368576d79ba2584717"
format_stamp: "Formatted at 2026-06-19 16:59:13 on dist-test-slave-vnbj"
server_key: "66d4fffd74bba34a27788c403894efbd"
server_key_iv: "f7e9e6fe9f4f2299d3b0364ecb02f4e1"
server_key_version: "encryptionkey@0"
I20260619 16:59:13.929379 30957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:59:13.948843 30957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:59:13.949182 30957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:59:13.949347 30957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:59:13.949602 30957 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:59:13.949965 30957 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:59:13.950021 30957 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:59:13.950062 30957 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:59:13.950091 30957 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:59:13.955966 30957 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.131:43283
I20260619 16:59:13.956009 31084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.131:43283 every 8 connection(s)
I20260619 16:59:13.956344 30957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
I20260619 16:59:13.961562 30674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:13.957083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:53975 (local address 127.20.231.190:36163)
0619 16:59:13.957152 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:13.957154 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:13.957410 (+ 256us) server_negotiation.cc:407] Connection header received
0619 16:59:13.957884 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:13.957885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:13.957895 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:13.957957 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:13.958606 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:13.959233 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:13.960197 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:13.960415 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:13.960472 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:13.961271 (+ 799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:59:13.961272 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:59:13.961272 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:59:13.961278 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:59:13.961291 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:59:13.961295 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:59:13.961340 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 16:59:13.961479 (+ 139us) server_negotiation.cc:299] Negotiation successful
0619 16:59:13.961501 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 16:59:13.961802 31087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:13.957081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36163 (local address 127.20.231.131:53975)
0619 16:59:13.957271 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:13.957314 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:13.957796 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:13.958010 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:13.958013 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:13.958021 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:13.958518 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:13.958523 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:13.959323 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:13.959324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:13.960117 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:13.960119 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:13.960493 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:13.960494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:13.960604 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:13.961112 (+ 508us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 16:59:13.961113 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:13.961140 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 16:59:13.961162 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:59:13.961385 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:59:13.961392 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:59:13.961395 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:59:13.961470 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 16:59:13.961559 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":94,"threads_started":1}
I20260619 16:59:13.962306 31085 heartbeater.cc:347] Connected to a master server at 127.20.231.190:36163
I20260619 16:59:13.962417 31085 heartbeater.cc:464] Registering TS with master...
I20260619 16:59:13.962601 31085 heartbeater.cc:511] Master 127.20.231.190:36163 requested a full tablet report, sending...
I20260619 16:59:13.962980 30635 ts_manager.cc:194] Registered new tserver with Master: 20654be425564e368576d79ba2584717 (127.20.231.131:43283)
I20260619 16:59:13.963306 30635 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.231.131:53975
I20260619 16:59:13.965844 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 30957
I20260619 16:59:13.965919 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance
I20260619 16:59:13.966111 21406 external_mini_cluster.cc:1428] Setting key 4cfed5d75e9189600d52a66a12bec597
I20260619 16:59:13.967271 21406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 16:59:13.969153 30674 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:13.968687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33910 (local address 127.20.231.190:36163)
0619 16:59:13.968803 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:13.968805 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:13.968844 (+ 39us) server_negotiation.cc:407] Connection header received
0619 16:59:13.968903 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:13.968904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:13.968912 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:13.968953 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:13.969063 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33910: BlockingRecv error: recv got EOF from 127.0.0.1:33910 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
I20260619 16:59:13.969386 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 30695
I20260619 16:59:13.974371 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 30826
I20260619 16:59:13.978881 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 30957
I20260619 16:59:13.983311 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 30604
2026-06-19T16:59:13Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (604 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-06-19T16:59:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:59:14Z Disabled control of system clock
I20260619 16:59:14.009892 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:46685
--webserver_interface=127.20.231.190
--webserver_port=0
--builtin_ntp_servers=127.20.231.148:35311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:46685
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 16:59:14.093209 31096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:59:14.093429 31096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:59:14.093451 31096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:59:14.094667 31096 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:59:14.094717 31096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:59:14.094736 31096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:59:14.094750 31096 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:59:14.094762 31096 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:59:14.096149 31096 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:35311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:46685
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:46685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:59:14.096359 31096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:59:14.096599 31096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:59:14.101521 31102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:59:14.101583 31101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:59:14.101745 31096 server_base.cc:1034] running on GCE node
W20260619 16:59:14.101570 31104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:59:14.102120 31096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:59:14.102420 31096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:59:14.103586 31096 hybrid_clock.cc:648] HybridClock initialized: now 1781888354103573 us; error 44 us; skew 500 ppm
I20260619 16:59:14.104897 31096 webserver.cc:460] Webserver started at http://127.20.231.190:41441/ using document root <none> and password file <none>
I20260619 16:59:14.105151 31096 fs_manager.cc:359] Metadata directory not provided
I20260619 16:59:14.105221 31096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:59:14.105373 31096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:59:14.106446 31096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "1165a442929a4e2c9787346525f70a9a"
format_stamp: "Formatted at 2026-06-19 16:59:14 on dist-test-slave-vnbj"
server_key: "9c2005ca3eda631a3c2a86c257ed6440"
server_key_iv: "0bfcfb82563a7f98a5df71f837b84109"
server_key_version: "encryptionkey@0"
I20260619 16:59:14.106817 31096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "1165a442929a4e2c9787346525f70a9a"
format_stamp: "Formatted at 2026-06-19 16:59:14 on dist-test-slave-vnbj"
server_key: "9c2005ca3eda631a3c2a86c257ed6440"
server_key_iv: "0bfcfb82563a7f98a5df71f837b84109"
server_key_version: "encryptionkey@0"
I20260619 16:59:14.108137 31096 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:59:14.108899 31110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:59:14.109063 31096 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:59:14.109167 31096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "1165a442929a4e2c9787346525f70a9a"
format_stamp: "Formatted at 2026-06-19 16:59:14 on dist-test-slave-vnbj"
server_key: "9c2005ca3eda631a3c2a86c257ed6440"
server_key_iv: "0bfcfb82563a7f98a5df71f837b84109"
server_key_version: "encryptionkey@0"
I20260619 16:59:14.109239 31096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:59:14.112675 31096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:59:14.112942 31096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:59:14.113087 31096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:59:14.117091 31096 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:46685
I20260619 16:59:14.117125 31162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:46685 every 8 connection(s)
I20260619 16:59:14.117466 31096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 16:59:14.118161 31163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:59:14.121045 31163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1165a442929a4e2c9787346525f70a9a: Bootstrap starting.
I20260619 16:59:14.121628 31163 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1165a442929a4e2c9787346525f70a9a: Neither blocks nor log segments found. Creating new log.
I20260619 16:59:14.121953 31163 log.cc:824] T 00000000000000000000000000000000 P 1165a442929a4e2c9787346525f70a9a: Log is configured to *not* fsync() on all Append() calls
I20260619 16:59:14.122663 31163 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1165a442929a4e2c9787346525f70a9a: No bootstrap required, opened a new log
I20260619 16:59:14.124159 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 31096
I20260619 16:59:14.124241 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 16:59:14.124439 21406 external_mini_cluster.cc:1428] Setting key b60a2fe014f049301600ace87dc74e6a
I20260619 16:59:14.124413 31163 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1165a442929a4e2c9787346525f70a9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1165a442929a4e2c9787346525f70a9a" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 46685 } }
I20260619 16:59:14.124604 31163 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1165a442929a4e2c9787346525f70a9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:59:14.124650 31163 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1165a442929a4e2c9787346525f70a9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1165a442929a4e2c9787346525f70a9a, State: Initialized, Role: FOLLOWER
I20260619 16:59:14.124799 31163 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1165a442929a4e2c9787346525f70a9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1165a442929a4e2c9787346525f70a9a" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 46685 } }
I20260619 16:59:14.124882 31163 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1165a442929a4e2c9787346525f70a9a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:59:14.124920 31163 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1165a442929a4e2c9787346525f70a9a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:59:14.124969 31163 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1165a442929a4e2c9787346525f70a9a [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:59:14.125628 31163 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1165a442929a4e2c9787346525f70a9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1165a442929a4e2c9787346525f70a9a" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 46685 } }
I20260619 16:59:14.126076 31163 leader_election.cc:302] T 00000000000000000000000000000000 P 1165a442929a4e2c9787346525f70a9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1165a442929a4e2c9787346525f70a9a; no voters:
I20260619 16:59:14.126268 31163 leader_election.cc:288] T 00000000000000000000000000000000 P 1165a442929a4e2c9787346525f70a9a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:59:14.126313 31168 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1165a442929a4e2c9787346525f70a9a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:59:14.126415 31168 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1165a442929a4e2c9787346525f70a9a [term 1 LEADER]: Becoming Leader. State: Replica: 1165a442929a4e2c9787346525f70a9a, State: Running, Role: LEADER
I20260619 16:59:14.126488 31168 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1165a442929a4e2c9787346525f70a9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1165a442929a4e2c9787346525f70a9a" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 46685 } }
I20260619 16:59:14.126528 31163 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1165a442929a4e2c9787346525f70a9a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:59:14.127091 31167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:14.124975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37930 (local address 127.20.231.190:46685)
0619 16:59:14.125247 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:14.125255 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:14.125280 (+ 25us) server_negotiation.cc:407] Connection header received
0619 16:59:14.125380 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:14.125385 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:14.125428 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:14.125551 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:14.126259 (+ 708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:59:14.126262 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:59:14.126262 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:59:14.126267 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:59:14.126305 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:59:14.126338 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:59:14.126446 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 16:59:14.126790 (+ 344us) server_negotiation.cc:299] Negotiation successful
0619 16:59:14.126866 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":115,"threads_started":1}
I20260619 16:59:14.127122 31170 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1165a442929a4e2c9787346525f70a9a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1165a442929a4e2c9787346525f70a9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1165a442929a4e2c9787346525f70a9a" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 46685 } } }
I20260619 16:59:14.127223 31169 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1165a442929a4e2c9787346525f70a9a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1165a442929a4e2c9787346525f70a9a. Latest consensus state: current_term: 1 leader_uuid: "1165a442929a4e2c9787346525f70a9a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1165a442929a4e2c9787346525f70a9a" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 46685 } } }
I20260619 16:59:14.127439 31169 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1165a442929a4e2c9787346525f70a9a [sys.catalog]: This master's current role is: LEADER
I20260619 16:59:14.127595 31170 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1165a442929a4e2c9787346525f70a9a [sys.catalog]: This master's current role is: LEADER
I20260619 16:59:14.128013 31179 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:59:14.128511 31179 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:59:14.130049 31179 catalog_manager.cc:1347] Generated new cluster ID: 0ac01ebc85c043cfbadf94bd570abe98
I20260619 16:59:14.130116 31179 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 16:59:14.147796 31179 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:59:14.148013 31179 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:59:14.151978 31179 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1165a442929a4e2c9787346525f70a9a: Generated new TSK 0
I20260619 16:59:14.152216 31179 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 16:59:14.155321 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.129:0
--local_ip_for_outbound_sockets=127.20.231.129
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:46685
--builtin_ntp_servers=127.20.231.148:35311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 16:59:14.238937 31187 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:59:14.239149 31187 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:59:14.239173 31187 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:59:14.240383 31187 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:59:14.240423 31187 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:59:14.240448 31187 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.129
I20260619 16:59:14.241772 31187 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:35311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:46685
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31187
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.129
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:59:14.242041 31187 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:59:14.242344 31187 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 16:59:14.248658 31187 server_base.cc:1034] running on GCE node
W20260619 16:59:14.248777 31195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:59:14.248642 31193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:59:14.248642 31192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:59:14.249197 31187 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:59:14.249468 31187 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:59:14.250638 31187 hybrid_clock.cc:648] HybridClock initialized: now 1781888354250618 us; error 38 us; skew 500 ppm
I20260619 16:59:14.252022 31187 webserver.cc:460] Webserver started at http://127.20.231.129:33991/ using document root <none> and password file <none>
I20260619 16:59:14.252283 31187 fs_manager.cc:359] Metadata directory not provided
I20260619 16:59:14.252388 31187 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:59:14.252537 31187 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:59:14.253620 31187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/data/instance:
uuid: "4fa6550341ee4e798d53e0c6f6906d89"
format_stamp: "Formatted at 2026-06-19 16:59:14 on dist-test-slave-vnbj"
server_key: "97718c987effb36f697bc9eb06b21cae"
server_key_iv: "06d46e29c55d55c9757c9744c6f5cee9"
server_key_version: "encryptionkey@0"
I20260619 16:59:14.253976 31187 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance:
uuid: "4fa6550341ee4e798d53e0c6f6906d89"
format_stamp: "Formatted at 2026-06-19 16:59:14 on dist-test-slave-vnbj"
server_key: "97718c987effb36f697bc9eb06b21cae"
server_key_iv: "06d46e29c55d55c9757c9744c6f5cee9"
server_key_version: "encryptionkey@0"
I20260619 16:59:14.255435 31187 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:59:14.256150 31201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:59:14.256306 31187 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260619 16:59:14.256415 31187 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/wal
uuid: "4fa6550341ee4e798d53e0c6f6906d89"
format_stamp: "Formatted at 2026-06-19 16:59:14 on dist-test-slave-vnbj"
server_key: "97718c987effb36f697bc9eb06b21cae"
server_key_iv: "06d46e29c55d55c9757c9744c6f5cee9"
server_key_version: "encryptionkey@0"
I20260619 16:59:14.256490 31187 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:59:14.260116 31187 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:59:14.260413 31187 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:59:14.260555 31187 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:59:14.260757 31187 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:59:14.261067 31187 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:59:14.261121 31187 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:59:14.261162 31187 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:59:14.261189 31187 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:59:14.266978 31187 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.129:46729
I20260619 16:59:14.267050 31314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.129:46729 every 8 connection(s)
I20260619 16:59:14.267444 31187 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
I20260619 16:59:14.270366 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 31187
I20260619 16:59:14.270470 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance
I20260619 16:59:14.270491 31167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:14.268211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:38789 (local address 127.20.231.190:46685)
0619 16:59:14.268273 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:14.268275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:14.268546 (+ 271us) server_negotiation.cc:407] Connection header received
0619 16:59:14.269001 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:14.269002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:14.269008 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:14.269063 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:14.270145 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:59:14.270146 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:59:14.270146 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:59:14.270150 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:59:14.270164 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:59:14.270168 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:59:14.270265 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 16:59:14.270401 (+ 136us) server_negotiation.cc:299] Negotiation successful
0619 16:59:14.270420 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 16:59:14.270684 31317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:14.268213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:46685 (local address 127.20.231.129:38789)
0619 16:59:14.268421 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:14.268454 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:14.268914 (+ 460us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:14.269117 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:14.269120 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:14.269129 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:14.270004 (+ 875us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 16:59:14.270005 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:14.270034 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 16:59:14.270063 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:59:14.270263 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:59:14.270270 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:59:14.270273 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:59:14.270397 (+ 124us) client_negotiation.cc:240] Negotiation successful
0619 16:59:14.270461 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":91,"threads_started":1}
I20260619 16:59:14.270706 21406 external_mini_cluster.cc:1428] Setting key bd5ba6b254d599454351e3c12c983684
I20260619 16:59:14.271287 31315 heartbeater.cc:347] Connected to a master server at 127.20.231.190:46685
I20260619 16:59:14.271384 31315 heartbeater.cc:464] Registering TS with master...
I20260619 16:59:14.271535 31315 heartbeater.cc:511] Master 127.20.231.190:46685 requested a full tablet report, sending...
I20260619 16:59:14.271777 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.130:0
--local_ip_for_outbound_sockets=127.20.231.130
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:46685
--builtin_ntp_servers=127.20.231.148:35311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260619 16:59:14.272079 31127 ts_manager.cc:194] Registered new tserver with Master: 4fa6550341ee4e798d53e0c6f6906d89 (127.20.231.129:46729)
W20260619 16:59:14.353297 31318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:59:14.353510 31318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:59:14.353533 31318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:59:14.354807 31318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:59:14.354877 31318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:59:14.354913 31318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.130
I20260619 16:59:14.356420 31318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:35311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:46685
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.130
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:59:14.356614 31318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:59:14.356868 31318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:59:14.362066 31324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:59:14.362066 31323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:59:14.362135 31318 server_base.cc:1034] running on GCE node
W20260619 16:59:14.362059 31326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:59:14.362491 31318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:59:14.362738 31318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:59:14.363902 31318 hybrid_clock.cc:648] HybridClock initialized: now 1781888354363894 us; error 41 us; skew 500 ppm
I20260619 16:59:14.365131 31318 webserver.cc:460] Webserver started at http://127.20.231.130:44851/ using document root <none> and password file <none>
I20260619 16:59:14.365396 31318 fs_manager.cc:359] Metadata directory not provided
I20260619 16:59:14.365468 31318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:59:14.365617 31318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:59:14.366684 31318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/data/instance:
uuid: "ca6e26527ac84b0c9af49eb99103b51f"
format_stamp: "Formatted at 2026-06-19 16:59:14 on dist-test-slave-vnbj"
server_key: "091551c5fd5ef025d29e9309d3e85ae4"
server_key_iv: "693ec683424e677dc7651a6a2e6f1e3c"
server_key_version: "encryptionkey@0"
I20260619 16:59:14.367116 31318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance:
uuid: "ca6e26527ac84b0c9af49eb99103b51f"
format_stamp: "Formatted at 2026-06-19 16:59:14 on dist-test-slave-vnbj"
server_key: "091551c5fd5ef025d29e9309d3e85ae4"
server_key_iv: "693ec683424e677dc7651a6a2e6f1e3c"
server_key_version: "encryptionkey@0"
I20260619 16:59:14.368369 31318 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:59:14.369046 31332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:59:14.369210 31318 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 16:59:14.369315 31318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/wal
uuid: "ca6e26527ac84b0c9af49eb99103b51f"
format_stamp: "Formatted at 2026-06-19 16:59:14 on dist-test-slave-vnbj"
server_key: "091551c5fd5ef025d29e9309d3e85ae4"
server_key_iv: "693ec683424e677dc7651a6a2e6f1e3c"
server_key_version: "encryptionkey@0"
I20260619 16:59:14.369403 31318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:59:14.372841 31318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:59:14.373116 31318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:59:14.373265 31318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:59:14.373476 31318 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:59:14.373785 31318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:59:14.373838 31318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:59:14.373879 31318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:59:14.373907 31318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:59:14.379979 31318 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.130:40527
I20260619 16:59:14.380024 31445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.130:40527 every 8 connection(s)
I20260619 16:59:14.380398 31318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
I20260619 16:59:14.383420 31167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:14.381252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:60771 (local address 127.20.231.190:46685)
0619 16:59:14.381300 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:14.381303 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:14.381511 (+ 208us) server_negotiation.cc:407] Connection header received
0619 16:59:14.381963 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:14.381965 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:14.381971 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:14.382034 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:14.383209 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:59:14.383210 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:59:14.383210 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:59:14.383214 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:59:14.383229 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:59:14.383233 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:59:14.383291 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 16:59:14.383354 (+ 63us) server_negotiation.cc:299] Negotiation successful
0619 16:59:14.383366 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 16:59:14.383632 31448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:14.381167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:46685 (local address 127.20.231.130:60771)
0619 16:59:14.381354 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:14.381395 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:14.381862 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:14.382083 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:14.382086 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:14.382100 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:14.383067 (+ 967us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 16:59:14.383068 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:14.383097 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 16:59:14.383124 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:59:14.383291 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:59:14.383293 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:59:14.383296 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:59:14.383350 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 16:59:14.383398 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":94,"threads_started":1}
I20260619 16:59:14.384261 31446 heartbeater.cc:347] Connected to a master server at 127.20.231.190:46685
I20260619 16:59:14.384363 31446 heartbeater.cc:464] Registering TS with master...
I20260619 16:59:14.384524 31446 heartbeater.cc:511] Master 127.20.231.190:46685 requested a full tablet report, sending...
I20260619 16:59:14.384974 31127 ts_manager.cc:194] Registered new tserver with Master: ca6e26527ac84b0c9af49eb99103b51f (127.20.231.130:40527)
I20260619 16:59:14.386827 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 31318
I20260619 16:59:14.386955 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance
I20260619 16:59:14.387112 21406 external_mini_cluster.cc:1428] Setting key 233f7befd774da0ff8b4b923f9c270ce
I20260619 16:59:14.388049 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.131:0
--local_ip_for_outbound_sockets=127.20.231.131
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:46685
--builtin_ntp_servers=127.20.231.148:35311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 16:59:14.469599 31449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:59:14.469810 31449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:59:14.469833 31449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:59:14.471022 31449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:59:14.471063 31449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:59:14.471088 31449 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.131
I20260619 16:59:14.472499 31449 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:35311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:46685
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.131
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:59:14.472703 31449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:59:14.472985 31449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 16:59:14.477962 31449 server_base.cc:1034] running on GCE node
W20260619 16:59:14.477981 31457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:59:14.477977 31455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:59:14.478152 31454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:59:14.478404 31449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:59:14.478663 31449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:59:14.479831 31449 hybrid_clock.cc:648] HybridClock initialized: now 1781888354479806 us; error 37 us; skew 500 ppm
I20260619 16:59:14.481140 31449 webserver.cc:460] Webserver started at http://127.20.231.131:44651/ using document root <none> and password file <none>
I20260619 16:59:14.481441 31449 fs_manager.cc:359] Metadata directory not provided
I20260619 16:59:14.481489 31449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:59:14.481665 31449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:59:14.482759 31449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/data/instance:
uuid: "bcb8b7e6af5b44d48f3d2d3d6b32ee45"
format_stamp: "Formatted at 2026-06-19 16:59:14 on dist-test-slave-vnbj"
server_key: "160734a80ab3666748112742a36a8d43"
server_key_iv: "2b25ec04013b138c344813bafbe58f62"
server_key_version: "encryptionkey@0"
I20260619 16:59:14.483207 31449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance:
uuid: "bcb8b7e6af5b44d48f3d2d3d6b32ee45"
format_stamp: "Formatted at 2026-06-19 16:59:14 on dist-test-slave-vnbj"
server_key: "160734a80ab3666748112742a36a8d43"
server_key_iv: "2b25ec04013b138c344813bafbe58f62"
server_key_version: "encryptionkey@0"
I20260619 16:59:14.484535 31449 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:59:14.485311 31463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:59:14.485507 31449 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:59:14.485579 31449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/wal
uuid: "bcb8b7e6af5b44d48f3d2d3d6b32ee45"
format_stamp: "Formatted at 2026-06-19 16:59:14 on dist-test-slave-vnbj"
server_key: "160734a80ab3666748112742a36a8d43"
server_key_iv: "2b25ec04013b138c344813bafbe58f62"
server_key_version: "encryptionkey@0"
I20260619 16:59:14.485670 31449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:59:14.489099 31449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:59:14.489382 31449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:59:14.489521 31449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:59:14.489717 31449 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:59:14.490025 31449 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:59:14.490056 31449 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:59:14.490108 31449 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:59:14.490134 31449 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:59:14.495986 31449 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.131:46367
I20260619 16:59:14.496076 31576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.131:46367 every 8 connection(s)
I20260619 16:59:14.496397 31449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
I20260619 16:59:14.499457 31167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:14.497141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:36297 (local address 127.20.231.190:46685)
0619 16:59:14.497193 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:14.497196 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:14.497436 (+ 240us) server_negotiation.cc:407] Connection header received
0619 16:59:14.497876 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:14.497877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:14.497882 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:14.497932 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:14.499228 (+ 1296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:59:14.499229 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:59:14.499229 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 16:59:14.499234 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:59:14.499249 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 16:59:14.499253 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:59:14.499330 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 16:59:14.499390 (+ 60us) server_negotiation.cc:299] Negotiation successful
0619 16:59:14.499403 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 16:59:14.499637 31579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:14.497142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:46685 (local address 127.20.231.131:36297)
0619 16:59:14.497320 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:14.497353 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:14.497792 (+ 439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:14.497990 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:14.497994 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:14.498001 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:14.499081 (+ 1080us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 16:59:14.499082 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:14.499111 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 16:59:14.499140 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:59:14.499330 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:59:14.499332 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:59:14.499335 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:59:14.499386 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 16:59:14.499431 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":80,"threads_started":1}
I20260619 16:59:14.500078 31577 heartbeater.cc:347] Connected to a master server at 127.20.231.190:46685
I20260619 16:59:14.500166 31577 heartbeater.cc:464] Registering TS with master...
I20260619 16:59:14.500288 31577 heartbeater.cc:511] Master 127.20.231.190:46685 requested a full tablet report, sending...
I20260619 16:59:14.500681 31127 ts_manager.cc:194] Registered new tserver with Master: bcb8b7e6af5b44d48f3d2d3d6b32ee45 (127.20.231.131:46367)
I20260619 16:59:14.502506 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 31449
I20260619 16:59:14.502581 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance
I20260619 16:59:14.502720 21406 external_mini_cluster.cc:1428] Setting key 3c2d1e8220994c4d623b0d688940a769
I20260619 16:59:14.503574 21406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 16:59:14.505263 31167 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 16:59:14.504848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37946 (local address 127.20.231.190:46685)
0619 16:59:14.504907 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:14.504908 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:14.504997 (+ 89us) server_negotiation.cc:407] Connection header received
0619 16:59:14.505049 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:14.505049 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:14.505053 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:14.505100 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:14.505182 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37946: BlockingRecv error: recv got EOF from 127.0.0.1:37946 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 16:59:14.505366 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 31187
I20260619 16:59:14.510447 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 31318
I20260619 16:59:14.514907 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 31449
I20260619 16:59:14.519186 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 31096
2026-06-19T16:59:14Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (535 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:59:14 dist-test-slave-vnbj krb5kdc[31587](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:59:14 dist-test-slave-vnbj krb5kdc[31587](info): set up 2 sockets
Jun 19 16:59:14 dist-test-slave-vnbj krb5kdc[31587](info): commencing operation
krb5kdc: starting...
W20260619 16:59:15.559983 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 16:59:15 dist-test-slave-vnbj krb5kdc[31587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:59:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:59:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.231.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
I20260619 16:59:15.686578 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:36957
--webserver_interface=127.20.231.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:43537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:36957
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:59:15.788942 31600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:59:15.789206 31600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:59:15.789247 31600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:59:15.790489 31600 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 16:59:15.790558 31600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:59:15.790584 31600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:59:15.790614 31600 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 16:59:15.790639 31600 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 16:59:15.792227 31600 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:43537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:36957
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:36957
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:59:15.792522 31600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:59:15.792764 31600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:59:15.797849 31605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:59:15.797901 31600 server_base.cc:1034] running on GCE node
W20260619 16:59:15.797909 31606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:59:15.797849 31608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:59:15.798274 31600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:59:15.798540 31600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:59:15.799679 31600 hybrid_clock.cc:648] HybridClock initialized: now 1781888355799648 us; error 50 us; skew 500 ppm
Jun 19 16:59:15 dist-test-slave-vnbj krb5kdc[31587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888355, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:59:15.802167 31600 init.cc:377] Logged in from keytab as kudu/127.20.231.190@KRBTEST.COM (short username kudu)
I20260619 16:59:15.802635 31600 webserver.cc:460] Webserver started at http://127.20.231.190:36743/ using document root <none> and password file <none>
I20260619 16:59:15.802945 31600 fs_manager.cc:359] Metadata directory not provided
I20260619 16:59:15.803022 31600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:59:15.803159 31600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:59:15.804224 31600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "5a257ea494ea4bbfa1f2128cb9f97012"
format_stamp: "Formatted at 2026-06-19 16:59:15 on dist-test-slave-vnbj"
server_key: "5d047a04058884a166c7ebcf40fe96c3"
server_key_iv: "ab52a041c29ecee3b32b6972b54c4bb0"
server_key_version: "encryptionkey@0"
I20260619 16:59:15.804625 31600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "5a257ea494ea4bbfa1f2128cb9f97012"
format_stamp: "Formatted at 2026-06-19 16:59:15 on dist-test-slave-vnbj"
server_key: "5d047a04058884a166c7ebcf40fe96c3"
server_key_iv: "ab52a041c29ecee3b32b6972b54c4bb0"
server_key_version: "encryptionkey@0"
I20260619 16:59:15.805943 31600 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:59:15.806672 31615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:59:15.806938 31600 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:59:15.807039 31600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "5a257ea494ea4bbfa1f2128cb9f97012"
format_stamp: "Formatted at 2026-06-19 16:59:15 on dist-test-slave-vnbj"
server_key: "5d047a04058884a166c7ebcf40fe96c3"
server_key_iv: "ab52a041c29ecee3b32b6972b54c4bb0"
server_key_version: "encryptionkey@0"
I20260619 16:59:15.807121 31600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:59:15.829093 31600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:59:15.831329 31600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:59:15.831507 31600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:59:15.835631 31600 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:36957
I20260619 16:59:15.835767 31667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:36957 every 8 connection(s)
I20260619 16:59:15.836076 31600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 16:59:15.837206 31668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:59:15.839764 31668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a257ea494ea4bbfa1f2128cb9f97012: Bootstrap starting.
I20260619 16:59:15.840472 31668 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a257ea494ea4bbfa1f2128cb9f97012: Neither blocks nor log segments found. Creating new log.
I20260619 16:59:15.840797 31668 log.cc:824] T 00000000000000000000000000000000 P 5a257ea494ea4bbfa1f2128cb9f97012: Log is configured to *not* fsync() on all Append() calls
I20260619 16:59:15.841566 31668 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a257ea494ea4bbfa1f2128cb9f97012: No bootstrap required, opened a new log
I20260619 16:59:15.842104 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 31600
I20260619 16:59:15.842183 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 16:59:15.842396 21406 external_mini_cluster.cc:1428] Setting key 772e502e2fa2ae8b4cedc1e56ad4bce9
I20260619 16:59:15.843978 31668 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a257ea494ea4bbfa1f2128cb9f97012 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a257ea494ea4bbfa1f2128cb9f97012" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36957 } }
I20260619 16:59:15.844179 31668 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a257ea494ea4bbfa1f2128cb9f97012 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:59:15.844226 31668 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a257ea494ea4bbfa1f2128cb9f97012 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a257ea494ea4bbfa1f2128cb9f97012, State: Initialized, Role: FOLLOWER
I20260619 16:59:15.844383 31668 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a257ea494ea4bbfa1f2128cb9f97012 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a257ea494ea4bbfa1f2128cb9f97012" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36957 } }
I20260619 16:59:15.844478 31668 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a257ea494ea4bbfa1f2128cb9f97012 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 16:59:15.844516 31668 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a257ea494ea4bbfa1f2128cb9f97012 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 16:59:15.844573 31668 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a257ea494ea4bbfa1f2128cb9f97012 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:59:15.845690 31668 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a257ea494ea4bbfa1f2128cb9f97012 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a257ea494ea4bbfa1f2128cb9f97012" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36957 } }
I20260619 16:59:15.845799 31668 leader_election.cc:302] T 00000000000000000000000000000000 P 5a257ea494ea4bbfa1f2128cb9f97012 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a257ea494ea4bbfa1f2128cb9f97012; no voters:
I20260619 16:59:15.846007 31668 leader_election.cc:288] T 00000000000000000000000000000000 P 5a257ea494ea4bbfa1f2128cb9f97012 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 16:59:15.846052 31673 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a257ea494ea4bbfa1f2128cb9f97012 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:59:15.846230 31673 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a257ea494ea4bbfa1f2128cb9f97012 [term 1 LEADER]: Becoming Leader. State: Replica: 5a257ea494ea4bbfa1f2128cb9f97012, State: Running, Role: LEADER
I20260619 16:59:15.846292 31668 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a257ea494ea4bbfa1f2128cb9f97012 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 16:59:15.846362 31673 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a257ea494ea4bbfa1f2128cb9f97012 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a257ea494ea4bbfa1f2128cb9f97012" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36957 } }
I20260619 16:59:15.846800 31674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a257ea494ea4bbfa1f2128cb9f97012 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a257ea494ea4bbfa1f2128cb9f97012" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a257ea494ea4bbfa1f2128cb9f97012" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36957 } } }
I20260619 16:59:15.846894 31674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a257ea494ea4bbfa1f2128cb9f97012 [sys.catalog]: This master's current role is: LEADER
I20260619 16:59:15.847146 31675 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a257ea494ea4bbfa1f2128cb9f97012 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a257ea494ea4bbfa1f2128cb9f97012. Latest consensus state: current_term: 1 leader_uuid: "5a257ea494ea4bbfa1f2128cb9f97012" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a257ea494ea4bbfa1f2128cb9f97012" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 36957 } } }
I20260619 16:59:15.847262 31675 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a257ea494ea4bbfa1f2128cb9f97012 [sys.catalog]: This master's current role is: LEADER
I20260619 16:59:15.847637 31680 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 16:59:15.848173 31680 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 16:59:15.849998 31680 catalog_manager.cc:1347] Generated new cluster ID: 7b7879ecbd29490692aeecf1977de5c0
I20260619 16:59:15.850054 31680 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 16:59:15 dist-test-slave-vnbj krb5kdc[31587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:59:15.855027 31672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:15.843037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43186 (local address 127.20.231.190:36957)
0619 16:59:15.843198 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:15.843207 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:15.843235 (+ 28us) server_negotiation.cc:407] Connection header received
0619 16:59:15.843952 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:15.843959 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:15.844009 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:15.844128 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:15.844991 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:15.846052 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:15.847171 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:15.847820 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:15.847927 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:15.851042 (+ 3115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:59:15.851046 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:59:15.851047 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:59:15.851058 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:59:15.853067 (+ 2009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:59:15.853443 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:59:15.853446 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:59:15.853446 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:59:15.853570 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:59:15.853734 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:59:15.853735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:59:15.853735 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:59:15.854035 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:59:15.854183 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0619 16:59:15.854605 (+ 422us) server_negotiation.cc:299] Negotiation successful
0619 16:59:15.854740 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260619 16:59:15.866164 31680 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 16:59:15.866741 31680 catalog_manager.cc:1504] Loading token signing keys...
I20260619 16:59:15.869915 31680 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a257ea494ea4bbfa1f2128cb9f97012: Generated new TSK 0
I20260619 16:59:15.870117 31680 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.231.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab.
Entry for principal kudu/127.20.231.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab.
I20260619 16:59:15.902885 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.129:0
--local_ip_for_outbound_sockets=127.20.231.129
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:36957
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:43537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:59:15.986048 31694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:59:15.986321 31694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:59:15.986359 31694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:59:15.987661 31694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:59:15.987730 31694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:59:15.987775 31694 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.129
I20260619 16:59:15.989326 31694 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:43537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:36957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.129
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:59:15.989624 31694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:59:15.989876 31694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:59:15.995095 31700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:59:15.995083 31702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:59:15.995083 31699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:59:15.995425 31694 server_base.cc:1034] running on GCE node
I20260619 16:59:15.995641 31694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:59:15.995951 31694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:59:15.997114 31694 hybrid_clock.cc:648] HybridClock initialized: now 1781888355997104 us; error 32 us; skew 500 ppm
Jun 19 16:59:15 dist-test-slave-vnbj krb5kdc[31587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888355, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:59:15.999889 31694 init.cc:377] Logged in from keytab as kudu/127.20.231.129@KRBTEST.COM (short username kudu)
I20260619 16:59:16.000353 31694 webserver.cc:460] Webserver started at http://127.20.231.129:42641/ using document root <none> and password file <none>
I20260619 16:59:16.000607 31694 fs_manager.cc:359] Metadata directory not provided
I20260619 16:59:16.000651 31694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:59:16.000811 31694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:59:16.001886 31694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/data/instance:
uuid: "e29f9333926146a0bd240ed63dd20ef9"
format_stamp: "Formatted at 2026-06-19 16:59:15 on dist-test-slave-vnbj"
server_key: "b0b4dffc555f24303fec8b20f0a45977"
server_key_iv: "f768141bf7d32bae34fb29fa7ff9ba8b"
server_key_version: "encryptionkey@0"
I20260619 16:59:16.002242 31694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance:
uuid: "e29f9333926146a0bd240ed63dd20ef9"
format_stamp: "Formatted at 2026-06-19 16:59:15 on dist-test-slave-vnbj"
server_key: "b0b4dffc555f24303fec8b20f0a45977"
server_key_iv: "f768141bf7d32bae34fb29fa7ff9ba8b"
server_key_version: "encryptionkey@0"
I20260619 16:59:16.003660 31694 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 16:59:16.004518 31709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:59:16.004799 31694 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:59:16.004895 31694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/wal
uuid: "e29f9333926146a0bd240ed63dd20ef9"
format_stamp: "Formatted at 2026-06-19 16:59:15 on dist-test-slave-vnbj"
server_key: "b0b4dffc555f24303fec8b20f0a45977"
server_key_iv: "f768141bf7d32bae34fb29fa7ff9ba8b"
server_key_version: "encryptionkey@0"
I20260619 16:59:16.004952 31694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:59:16.013736 31694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:59:16.015833 31694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:59:16.015947 31694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:59:16.016137 31694 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:59:16.016433 31694 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:59:16.016469 31694 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:59:16.016491 31694 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:59:16.016543 31694 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:59:16.022420 31694 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.129:39305
I20260619 16:59:16.022516 31822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.129:39305 every 8 connection(s)
I20260619 16:59:16.022912 31694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
I20260619 16:59:16.028250 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 31694
I20260619 16:59:16.028384 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance
I20260619 16:59:16.028621 21406 external_mini_cluster.cc:1428] Setting key 9a9ef5d67f750e1a15c6a10ada8e735d
Jun 19 16:59:16 dist-test-slave-vnbj krb5kdc[31587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888355, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:59:16.033571 31672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:16.023684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:50223 (local address 127.20.231.190:36957)
0619 16:59:16.023810 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:16.023813 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:16.023936 (+ 123us) server_negotiation.cc:407] Connection header received
0619 16:59:16.024919 (+ 983us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:16.024920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:16.024932 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:16.024994 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:16.025575 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:16.026064 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:16.027181 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:16.027385 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:16.027450 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:16.029947 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:59:16.029949 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:59:16.029949 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:59:16.029957 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:59:16.031872 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:59:16.032410 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:59:16.032411 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:59:16.032411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:59:16.032477 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:59:16.032796 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:59:16.032797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:59:16.032797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:59:16.032920 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:59:16.032986 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 16:59:16.033402 (+ 416us) server_negotiation.cc:299] Negotiation successful
0619 16:59:16.033503 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260619 16:59:16.033788 31825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:16.023639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36957 (local address 127.20.231.129:50223)
0619 16:59:16.023810 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:16.023852 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:16.024827 (+ 975us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:16.025050 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:16.025053 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:16.025314 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:16.025503 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:16.025508 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:16.026151 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:16.026151 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:16.027034 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:16.027037 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:16.027467 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:16.027467 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:16.027566 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:16.028168 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:16.028176 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:16.029838 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:59:16.031977 (+ 2139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:59:16.031984 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:59:16.031988 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:59:16.032318 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:59:16.032556 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:59:16.032557 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:59:16.032558 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:59:16.032705 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:59:16.032999 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:59:16.033005 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:59:16.033183 (+ 178us) client_negotiation.cc:769] Sending connection context
0619 16:59:16.033315 (+ 132us) client_negotiation.cc:240] Negotiation successful
0619 16:59:16.033511 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":100,"threads_started":1}
I20260619 16:59:16.034296 31823 heartbeater.cc:347] Connected to a master server at 127.20.231.190:36957
I20260619 16:59:16.034412 31823 heartbeater.cc:464] Registering TS with master...
I20260619 16:59:16.034621 31823 heartbeater.cc:511] Master 127.20.231.190:36957 requested a full tablet report, sending...
I20260619 16:59:16.035254 31632 ts_manager.cc:194] Registered new tserver with Master: e29f9333926146a0bd240ed63dd20ef9 (127.20.231.129:39305)
I20260619 16:59:16.036221 31632 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.129@KRBTEST.COM'} at 127.20.231.129:50223
WARNING: no policy specified for kudu/127.20.231.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab.
Entry for principal kudu/127.20.231.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab.
I20260619 16:59:16.058470 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.130:0
--local_ip_for_outbound_sockets=127.20.231.130
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:36957
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:43537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:59:16.140782 31828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:59:16.141049 31828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:59:16.141093 31828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:59:16.142314 31828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:59:16.142397 31828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:59:16.142443 31828 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.130
I20260619 16:59:16.143978 31828 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:43537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:36957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.130
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:59:16.144281 31828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:59:16.144531 31828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:59:16.150101 31836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:59:16.150117 31833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:59:16.150101 31834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:59:16.150478 31828 server_base.cc:1034] running on GCE node
I20260619 16:59:16.150717 31828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:59:16.151072 31828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:59:16.152256 31828 hybrid_clock.cc:648] HybridClock initialized: now 1781888356152243 us; error 35 us; skew 500 ppm
Jun 19 16:59:16 dist-test-slave-vnbj krb5kdc[31587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888356, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:59:16.155025 31828 init.cc:377] Logged in from keytab as kudu/127.20.231.130@KRBTEST.COM (short username kudu)
I20260619 16:59:16.155539 31828 webserver.cc:460] Webserver started at http://127.20.231.130:39871/ using document root <none> and password file <none>
I20260619 16:59:16.155781 31828 fs_manager.cc:359] Metadata directory not provided
I20260619 16:59:16.155853 31828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:59:16.156006 31828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:59:16.157100 31828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/data/instance:
uuid: "150f7cccc6444c1181216961c7577199"
format_stamp: "Formatted at 2026-06-19 16:59:16 on dist-test-slave-vnbj"
server_key: "fca759ed01a98db8aa55f55e5133802c"
server_key_iv: "ead4d9ee24e25676359ca3d24c7bd265"
server_key_version: "encryptionkey@0"
I20260619 16:59:16.157476 31828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance:
uuid: "150f7cccc6444c1181216961c7577199"
format_stamp: "Formatted at 2026-06-19 16:59:16 on dist-test-slave-vnbj"
server_key: "fca759ed01a98db8aa55f55e5133802c"
server_key_iv: "ead4d9ee24e25676359ca3d24c7bd265"
server_key_version: "encryptionkey@0"
I20260619 16:59:16.158975 31828 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 16:59:16.160046 31843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:59:16.160286 31828 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 16:59:16.160395 31828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/wal
uuid: "150f7cccc6444c1181216961c7577199"
format_stamp: "Formatted at 2026-06-19 16:59:16 on dist-test-slave-vnbj"
server_key: "fca759ed01a98db8aa55f55e5133802c"
server_key_iv: "ead4d9ee24e25676359ca3d24c7bd265"
server_key_version: "encryptionkey@0"
I20260619 16:59:16.160490 31828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:59:16.171790 31828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:59:16.173776 31828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:59:16.173954 31828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:59:16.174196 31828 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:59:16.174526 31828 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:59:16.174582 31828 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:59:16.174623 31828 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:59:16.174652 31828 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:59:16.180696 31828 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.130:46529
I20260619 16:59:16.180742 31956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.130:46529 every 8 connection(s)
I20260619 16:59:16.181084 31828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
I20260619 16:59:16.183605 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 31828
I20260619 16:59:16.183703 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance
I20260619 16:59:16.183911 21406 external_mini_cluster.cc:1428] Setting key d68d73c72b83a792807fdf747b19aa06
Jun 19 16:59:16 dist-test-slave-vnbj krb5kdc[31587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888356, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:59:16.192382 31672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:16.181866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:42609 (local address 127.20.231.190:36957)
0619 16:59:16.181927 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:16.181930 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:16.182095 (+ 165us) server_negotiation.cc:407] Connection header received
0619 16:59:16.182568 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:16.182569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:16.182578 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:16.182637 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:16.183770 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:16.184383 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:16.186039 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:16.186230 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:16.186322 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:16.188884 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:59:16.188886 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:59:16.188886 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:59:16.188895 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:59:16.190674 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:59:16.191197 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:59:16.191198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:59:16.191198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:59:16.191274 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:59:16.191613 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:59:16.191615 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:59:16.191615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:59:16.191769 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:59:16.191843 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 16:59:16.192132 (+ 289us) server_negotiation.cc:299] Negotiation successful
0619 16:59:16.192241 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 16:59:16.192456 31959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:16.181812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36957 (local address 127.20.231.130:42609)
0619 16:59:16.181968 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:16.182010 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:16.182488 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:16.182689 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:16.182692 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:16.183317 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:16.183619 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:16.183625 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:16.184494 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:16.184496 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:16.185939 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:16.185942 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:16.186327 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:16.186329 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:16.186466 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:16.187062 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:16.187071 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:16.188777 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:59:16.190785 (+ 2008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:59:16.190792 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:59:16.190793 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:59:16.191115 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:59:16.191390 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:59:16.191391 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:59:16.191391 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:59:16.191525 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:59:16.191838 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:59:16.191843 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:59:16.191967 (+ 124us) client_negotiation.cc:769] Sending connection context
0619 16:59:16.192060 (+ 93us) client_negotiation.cc:240] Negotiation successful
0619 16:59:16.192212 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":89,"threads_started":1}
I20260619 16:59:16.193053 31957 heartbeater.cc:347] Connected to a master server at 127.20.231.190:36957
I20260619 16:59:16.193147 31957 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.20.231.131@KRBTEST.COM; defaulting to no policy
I20260619 16:59:16.193351 31957 heartbeater.cc:511] Master 127.20.231.190:36957 requested a full tablet report, sending...
I20260619 16:59:16.193758 31632 ts_manager.cc:194] Registered new tserver with Master: 150f7cccc6444c1181216961c7577199 (127.20.231.130:46529)
I20260619 16:59:16.194140 31632 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.130@KRBTEST.COM'} at 127.20.231.130:42609
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab.
Entry for principal kudu/127.20.231.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab.
I20260619 16:59:16.211673 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.131:0
--local_ip_for_outbound_sockets=127.20.231.131
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:36957
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:43537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 16:59:16.292721 31962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 16:59:16.292989 31962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 16:59:16.293030 31962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 16:59:16.294203 31962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 16:59:16.294293 31962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 16:59:16.294339 31962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.131
I20260619 16:59:16.295938 31962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:43537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:36957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.131
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 16:59:16.296231 31962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 16:59:16.296480 31962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 16:59:16.301990 31967 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:59:16.301992 31968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 16:59:16.302109 31970 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 16:59:16.302170 31962 server_base.cc:1034] running on GCE node
I20260619 16:59:16.302523 31962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 16:59:16.302771 31962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 16:59:16.303918 31962 hybrid_clock.cc:648] HybridClock initialized: now 1781888356303904 us; error 27 us; skew 500 ppm
Jun 19 16:59:16 dist-test-slave-vnbj krb5kdc[31587](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888356, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 16:59:16.306645 31962 init.cc:377] Logged in from keytab as kudu/127.20.231.131@KRBTEST.COM (short username kudu)
I20260619 16:59:16.307264 31962 webserver.cc:460] Webserver started at http://127.20.231.131:36405/ using document root <none> and password file <none>
I20260619 16:59:16.307534 31962 fs_manager.cc:359] Metadata directory not provided
I20260619 16:59:16.307581 31962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 16:59:16.307677 31962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 16:59:16.308806 31962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/data/instance:
uuid: "ae748c996841450bb27312ae24e45e13"
format_stamp: "Formatted at 2026-06-19 16:59:16 on dist-test-slave-vnbj"
server_key: "8dda0deec230f5867de3f725601f1e21"
server_key_iv: "4fecdb70cc0bc9ea2f8dd353a6230148"
server_key_version: "encryptionkey@0"
I20260619 16:59:16.309160 31962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance:
uuid: "ae748c996841450bb27312ae24e45e13"
format_stamp: "Formatted at 2026-06-19 16:59:16 on dist-test-slave-vnbj"
server_key: "8dda0deec230f5867de3f725601f1e21"
server_key_iv: "4fecdb70cc0bc9ea2f8dd353a6230148"
server_key_version: "encryptionkey@0"
I20260619 16:59:16.310590 31962 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 16:59:16.311468 31977 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 16:59:16.311653 31962 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 16:59:16.311741 31962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/wal
uuid: "ae748c996841450bb27312ae24e45e13"
format_stamp: "Formatted at 2026-06-19 16:59:16 on dist-test-slave-vnbj"
server_key: "8dda0deec230f5867de3f725601f1e21"
server_key_iv: "4fecdb70cc0bc9ea2f8dd353a6230148"
server_key_version: "encryptionkey@0"
I20260619 16:59:16.311816 31962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 16:59:16.353991 31962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 16:59:16.356421 31962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 16:59:16.356537 31962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 16:59:16.356720 31962 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 16:59:16.357054 31962 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 16:59:16.357090 31962 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 16:59:16.357134 31962 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 16:59:16.357161 31962 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 16:59:16.363088 31962 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.131:38021
I20260619 16:59:16.363170 32090 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.131:38021 every 8 connection(s)
I20260619 16:59:16.363551 31962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
I20260619 16:59:16.367096 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 31962
I20260619 16:59:16.367197 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance
I20260619 16:59:16.367374 21406 external_mini_cluster.cc:1428] Setting key a7f027c4e81adfac57c9dd0f4a35340b
Jun 19 16:59:16 dist-test-slave-vnbj krb5kdc[31587](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888356, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 16:59:16.373366 31672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:16.364333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:37937 (local address 127.20.231.190:36957)
0619 16:59:16.364385 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:16.364387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:16.364577 (+ 190us) server_negotiation.cc:407] Connection header received
0619 16:59:16.365246 (+ 669us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:16.365247 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:16.365258 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:16.365321 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:16.366242 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:16.366741 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:16.368179 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:16.368419 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:16.368476 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:16.370786 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:59:16.370787 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:59:16.370787 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:59:16.370794 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:59:16.372042 (+ 1248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:59:16.372505 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:59:16.372506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:59:16.372506 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:59:16.372554 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:59:16.372808 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:59:16.372809 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:59:16.372809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:59:16.372909 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:59:16.372961 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 16:59:16.373244 (+ 283us) server_negotiation.cc:299] Negotiation successful
0619 16:59:16.373303 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 16:59:16.373680 32093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:16.364303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:36957 (local address 127.20.231.131:37937)
0619 16:59:16.364462 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:16.364505 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:16.365115 (+ 610us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:16.365376 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:16.365380 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:16.365958 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0619 16:59:16.366155 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:16.366159 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:16.366828 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:16.366829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:16.368099 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:16.368101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:16.368492 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:16.368493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:16.368603 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:16.369170 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 16:59:16.369179 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 16:59:16.370697 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 16:59:16.372145 (+ 1448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:59:16.372154 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:59:16.372156 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:59:16.372431 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:59:16.372616 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 16:59:16.372616 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 16:59:16.372617 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 16:59:16.372746 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 16:59:16.372968 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 16:59:16.372972 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 16:59:16.373092 (+ 120us) client_negotiation.cc:769] Sending connection context
0619 16:59:16.373209 (+ 117us) client_negotiation.cc:240] Negotiation successful
0619 16:59:16.373363 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
I20260619 16:59:16.374322 32091 heartbeater.cc:347] Connected to a master server at 127.20.231.190:36957
I20260619 16:59:16.374450 32091 heartbeater.cc:464] Registering TS with master...
I20260619 16:59:16.374652 32091 heartbeater.cc:511] Master 127.20.231.190:36957 requested a full tablet report, sending...
I20260619 16:59:16.375164 31632 ts_manager.cc:194] Registered new tserver with Master: ae748c996841450bb27312ae24e45e13 (127.20.231.131:38021)
I20260619 16:59:16.375535 31632 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.131@KRBTEST.COM'} at 127.20.231.131:37937
I20260619 16:59:16.379088 21406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 16:59:16.386485 31672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:16.380338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43192 (local address 127.20.231.190:36957)
0619 16:59:16.380384 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:16.380386 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:16.380484 (+ 98us) server_negotiation.cc:407] Connection header received
0619 16:59:16.380538 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:16.380539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:16.380547 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:16.380596 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 16:59:16.381118 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:16.381585 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:16.382489 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:16.382664 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:16.382715 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:16.384177 (+ 1462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 16:59:16.384178 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 16:59:16.384178 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 16:59:16.384184 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 16:59:16.385375 (+ 1191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:59:16.385703 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:59:16.385704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:59:16.385704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:59:16.385797 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 16:59:16.386020 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 16:59:16.386021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 16:59:16.386021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 16:59:16.386160 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 16:59:16.386246 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 16:59:16.386363 (+ 117us) server_negotiation.cc:299] Negotiation successful
0619 16:59:16.386420 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 16:59:16.388190 31632 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43192:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 16:59:16.389811 31632 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 16:59:16.396616 32104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:16.392320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37322 (local address 127.20.231.129:39305)
0619 16:59:16.392521 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:16.392526 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:16.392818 (+ 292us) server_negotiation.cc:407] Connection header received
0619 16:59:16.392945 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:16.392947 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:16.392962 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:16.393022 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:59:16.393252 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:16.394399 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:16.395590 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:16.396152 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:16.396263 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:16.396326 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 16:59:16.396471 (+ 145us) server_negotiation.cc:299] Negotiation successful
0619 16:59:16.396524 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":89,"threads_started":1}
I20260619 16:59:16.396720 32102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:16.392146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:38021 (local address 127.0.0.1:46032)
0619 16:59:16.392319 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:16.392362 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:16.392424 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:16.392961 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:16.392965 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:16.392967 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:59:16.393042 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:16.393044 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:16.394477 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:16.394478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:16.395616 (+ 1138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:16.395618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:16.396483 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:16.396484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:16.396552 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:16.396557 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 16:59:16.396587 (+ 30us) client_negotiation.cc:240] Negotiation successful
0619 16:59:16.396627 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":114,"threads_started":1}
I20260619 16:59:16.396677 32106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:16.392532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48064 (local address 127.20.231.130:46529)
0619 16:59:16.392713 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:16.392718 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:16.392876 (+ 158us) server_negotiation.cc:407] Connection header received
0619 16:59:16.392922 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:16.392924 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:16.392940 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:16.393011 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:59:16.393240 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:16.393938 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:16.395519 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:16.396327 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:16.396416 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:16.396476 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 16:59:16.396549 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 16:59:16.396595 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":81,"threads_started":1}
I20260619 16:59:16.396871 32103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:16.392415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:46529 (local address 127.0.0.1:48064)
0619 16:59:16.392752 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:16.392762 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:16.392819 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:16.393094 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:16.393095 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:16.393097 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:59:16.393156 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:16.393158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:16.394039 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:16.394040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:16.395416 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:16.395419 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:16.396413 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:16.396414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:16.396474 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:16.396478 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:59:16.396534 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 16:59:16.396592 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":77,"threads_started":1}
I20260619 16:59:16.397127 32105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:16.392180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:39305 (local address 127.0.0.1:37322)
0619 16:59:16.392713 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:16.392725 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:16.392848 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:16.393123 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:16.393124 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:16.393126 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:59:16.393169 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:16.393171 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:16.394512 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:16.394512 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:16.395438 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:16.395440 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:16.396263 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:16.396264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:16.396343 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:16.396358 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 16:59:16.397021 (+ 663us) client_negotiation.cc:240] Negotiation successful
0619 16:59:16.397069 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":96,"threads_started":1}
I20260619 16:59:16.397181 32107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:16.392557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46032 (local address 127.20.231.131:38021)
0619 16:59:16.392771 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:16.392775 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:16.392786 (+ 11us) server_negotiation.cc:407] Connection header received
0619 16:59:16.392828 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:16.392831 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:16.392848 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:16.393118 (+ 270us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:59:16.393181 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:16.394358 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:16.395712 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:16.396388 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:16.396940 (+ 552us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:16.397011 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 16:59:16.397037 (+ 26us) server_negotiation.cc:299] Negotiation successful
0619 16:59:16.397087 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":101,"threads_started":1}
I20260619 16:59:16.397644 31891 tablet_service.cc:1467] Processing CreateTablet for tablet a7319100c4d14a0ab4048e8330d24df4 (DEFAULT_TABLE table=test-table [id=6868d8d1c60c4d69b64e464f29eab142]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:59:16.397850 32025 tablet_service.cc:1467] Processing CreateTablet for tablet a7319100c4d14a0ab4048e8330d24df4 (DEFAULT_TABLE table=test-table [id=6868d8d1c60c4d69b64e464f29eab142]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:59:16.397934 31757 tablet_service.cc:1467] Processing CreateTablet for tablet a7319100c4d14a0ab4048e8330d24df4 (DEFAULT_TABLE table=test-table [id=6868d8d1c60c4d69b64e464f29eab142]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 16:59:16.398032 31891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7319100c4d14a0ab4048e8330d24df4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:59:16.398188 32025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7319100c4d14a0ab4048e8330d24df4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:59:16.398211 31757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a7319100c4d14a0ab4048e8330d24df4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 16:59:16.400799 32108 tablet_bootstrap.cc:492] T a7319100c4d14a0ab4048e8330d24df4 P 150f7cccc6444c1181216961c7577199: Bootstrap starting.
I20260619 16:59:16.401510 32109 tablet_bootstrap.cc:492] T a7319100c4d14a0ab4048e8330d24df4 P ae748c996841450bb27312ae24e45e13: Bootstrap starting.
I20260619 16:59:16.401710 32108 tablet_bootstrap.cc:654] T a7319100c4d14a0ab4048e8330d24df4 P 150f7cccc6444c1181216961c7577199: Neither blocks nor log segments found. Creating new log.
I20260619 16:59:16.402212 32109 tablet_bootstrap.cc:654] T a7319100c4d14a0ab4048e8330d24df4 P ae748c996841450bb27312ae24e45e13: Neither blocks nor log segments found. Creating new log.
I20260619 16:59:16.402374 32108 log.cc:824] T a7319100c4d14a0ab4048e8330d24df4 P 150f7cccc6444c1181216961c7577199: Log is configured to *not* fsync() on all Append() calls
I20260619 16:59:16.402758 32109 log.cc:824] T a7319100c4d14a0ab4048e8330d24df4 P ae748c996841450bb27312ae24e45e13: Log is configured to *not* fsync() on all Append() calls
I20260619 16:59:16.403122 32110 tablet_bootstrap.cc:492] T a7319100c4d14a0ab4048e8330d24df4 P e29f9333926146a0bd240ed63dd20ef9: Bootstrap starting.
I20260619 16:59:16.403208 32108 tablet_bootstrap.cc:492] T a7319100c4d14a0ab4048e8330d24df4 P 150f7cccc6444c1181216961c7577199: No bootstrap required, opened a new log
I20260619 16:59:16.403282 32108 ts_tablet_manager.cc:1397] T a7319100c4d14a0ab4048e8330d24df4 P 150f7cccc6444c1181216961c7577199: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 16:59:16.403498 32109 tablet_bootstrap.cc:492] T a7319100c4d14a0ab4048e8330d24df4 P ae748c996841450bb27312ae24e45e13: No bootstrap required, opened a new log
I20260619 16:59:16.403548 32109 ts_tablet_manager.cc:1397] T a7319100c4d14a0ab4048e8330d24df4 P ae748c996841450bb27312ae24e45e13: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 16:59:16.404934 32108 raft_consensus.cc:348] T a7319100c4d14a0ab4048e8330d24df4 P 150f7cccc6444c1181216961c7577199 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae748c996841450bb27312ae24e45e13" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 38021 } } peers { permanent_uuid: "e29f9333926146a0bd240ed63dd20ef9" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 39305 } } peers { permanent_uuid: "150f7cccc6444c1181216961c7577199" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46529 } }
I20260619 16:59:16.405082 32108 raft_consensus.cc:374] T a7319100c4d14a0ab4048e8330d24df4 P 150f7cccc6444c1181216961c7577199 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:59:16.405126 32108 raft_consensus.cc:729] T a7319100c4d14a0ab4048e8330d24df4 P 150f7cccc6444c1181216961c7577199 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 150f7cccc6444c1181216961c7577199, State: Initialized, Role: FOLLOWER
I20260619 16:59:16.405045 32109 raft_consensus.cc:348] T a7319100c4d14a0ab4048e8330d24df4 P ae748c996841450bb27312ae24e45e13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae748c996841450bb27312ae24e45e13" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 38021 } } peers { permanent_uuid: "e29f9333926146a0bd240ed63dd20ef9" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 39305 } } peers { permanent_uuid: "150f7cccc6444c1181216961c7577199" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46529 } }
I20260619 16:59:16.405206 32109 raft_consensus.cc:374] T a7319100c4d14a0ab4048e8330d24df4 P ae748c996841450bb27312ae24e45e13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:59:16.405262 32109 raft_consensus.cc:729] T a7319100c4d14a0ab4048e8330d24df4 P ae748c996841450bb27312ae24e45e13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae748c996841450bb27312ae24e45e13, State: Initialized, Role: FOLLOWER
I20260619 16:59:16.405241 32108 consensus_queue.cc:260] T a7319100c4d14a0ab4048e8330d24df4 P 150f7cccc6444c1181216961c7577199 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae748c996841450bb27312ae24e45e13" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 38021 } } peers { permanent_uuid: "e29f9333926146a0bd240ed63dd20ef9" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 39305 } } peers { permanent_uuid: "150f7cccc6444c1181216961c7577199" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46529 } }
I20260619 16:59:16.405373 32109 consensus_queue.cc:260] T a7319100c4d14a0ab4048e8330d24df4 P ae748c996841450bb27312ae24e45e13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae748c996841450bb27312ae24e45e13" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 38021 } } peers { permanent_uuid: "e29f9333926146a0bd240ed63dd20ef9" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 39305 } } peers { permanent_uuid: "150f7cccc6444c1181216961c7577199" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46529 } }
I20260619 16:59:16.405508 32108 ts_tablet_manager.cc:1428] T a7319100c4d14a0ab4048e8330d24df4 P 150f7cccc6444c1181216961c7577199: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 16:59:16.405550 31957 heartbeater.cc:503] Master 127.20.231.190:36957 was elected leader, sending a full tablet report...
I20260619 16:59:16.405638 32109 ts_tablet_manager.cc:1428] T a7319100c4d14a0ab4048e8330d24df4 P ae748c996841450bb27312ae24e45e13: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 16:59:16.405647 32091 heartbeater.cc:503] Master 127.20.231.190:36957 was elected leader, sending a full tablet report...
I20260619 16:59:16.406292 32110 tablet_bootstrap.cc:654] T a7319100c4d14a0ab4048e8330d24df4 P e29f9333926146a0bd240ed63dd20ef9: Neither blocks nor log segments found. Creating new log.
I20260619 16:59:16.406947 32110 log.cc:824] T a7319100c4d14a0ab4048e8330d24df4 P e29f9333926146a0bd240ed63dd20ef9: Log is configured to *not* fsync() on all Append() calls
I20260619 16:59:16.407737 32110 tablet_bootstrap.cc:492] T a7319100c4d14a0ab4048e8330d24df4 P e29f9333926146a0bd240ed63dd20ef9: No bootstrap required, opened a new log
I20260619 16:59:16.407804 32110 ts_tablet_manager.cc:1397] T a7319100c4d14a0ab4048e8330d24df4 P e29f9333926146a0bd240ed63dd20ef9: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260619 16:59:16.409456 32110 raft_consensus.cc:348] T a7319100c4d14a0ab4048e8330d24df4 P e29f9333926146a0bd240ed63dd20ef9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae748c996841450bb27312ae24e45e13" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 38021 } } peers { permanent_uuid: "e29f9333926146a0bd240ed63dd20ef9" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 39305 } } peers { permanent_uuid: "150f7cccc6444c1181216961c7577199" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46529 } }
I20260619 16:59:16.409580 32110 raft_consensus.cc:374] T a7319100c4d14a0ab4048e8330d24df4 P e29f9333926146a0bd240ed63dd20ef9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 16:59:16.409601 32110 raft_consensus.cc:729] T a7319100c4d14a0ab4048e8330d24df4 P e29f9333926146a0bd240ed63dd20ef9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e29f9333926146a0bd240ed63dd20ef9, State: Initialized, Role: FOLLOWER
I20260619 16:59:16.409713 32110 consensus_queue.cc:260] T a7319100c4d14a0ab4048e8330d24df4 P e29f9333926146a0bd240ed63dd20ef9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae748c996841450bb27312ae24e45e13" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 38021 } } peers { permanent_uuid: "e29f9333926146a0bd240ed63dd20ef9" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 39305 } } peers { permanent_uuid: "150f7cccc6444c1181216961c7577199" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46529 } }
I20260619 16:59:16.410001 32110 ts_tablet_manager.cc:1428] T a7319100c4d14a0ab4048e8330d24df4 P e29f9333926146a0bd240ed63dd20ef9: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260619 16:59:16.410019 31823 heartbeater.cc:503] Master 127.20.231.190:36957 was elected leader, sending a full tablet report...
W20260619 16:59:16.431656 31958 tablet.cc:2369] T a7319100c4d14a0ab4048e8330d24df4 P 150f7cccc6444c1181216961c7577199: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:59:16.502269 32116 raft_consensus.cc:482] T a7319100c4d14a0ab4048e8330d24df4 P e29f9333926146a0bd240ed63dd20ef9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 16:59:16.502404 32116 raft_consensus.cc:504] T a7319100c4d14a0ab4048e8330d24df4 P e29f9333926146a0bd240ed63dd20ef9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae748c996841450bb27312ae24e45e13" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 38021 } } peers { permanent_uuid: "e29f9333926146a0bd240ed63dd20ef9" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 39305 } } peers { permanent_uuid: "150f7cccc6444c1181216961c7577199" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46529 } }
I20260619 16:59:16.502938 32116 leader_election.cc:288] T a7319100c4d14a0ab4048e8330d24df4 P e29f9333926146a0bd240ed63dd20ef9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae748c996841450bb27312ae24e45e13 (127.20.231.131:38021), 150f7cccc6444c1181216961c7577199 (127.20.231.130:46529)
I20260619 16:59:16.505926 32107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:16.503268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:44751 (local address 127.20.231.131:38021)
0619 16:59:16.503325 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:16.503327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:16.503336 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:59:16.503366 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:16.503367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:16.503376 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:16.503469 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:59:16.503633 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:16.504106 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:16.505157 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:16.505555 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:16.505643 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:16.505652 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 16:59:16.505819 (+ 167us) server_negotiation.cc:299] Negotiation successful
0619 16:59:16.505856 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 16:59:16.505928 31825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:16.503151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:38021 (local address 127.20.231.129:44751)
0619 16:59:16.503219 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:16.503232 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:16.503282 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:16.503473 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:16.503474 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:16.503476 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:59:16.503551 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:16.503553 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:16.504193 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:16.504194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:16.505043 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:16.505045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:16.505641 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:16.505642 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:16.505729 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:16.505732 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 16:59:16.505809 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 16:59:16.505852 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260619 16:59:16.505934 32106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:16.503274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:43163 (local address 127.20.231.130:46529)
0619 16:59:16.503319 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:16.503322 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:16.503510 (+ 188us) server_negotiation.cc:407] Connection header received
0619 16:59:16.503570 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:16.503571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:16.503581 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:16.503638 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 16:59:16.503797 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:16.504278 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:16.505165 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:16.505573 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:16.505660 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:16.505668 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 16:59:16.505829 (+ 161us) server_negotiation.cc:299] Negotiation successful
0619 16:59:16.505869 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 16:59:16.505949 32117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:16.503156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:46529 (local address 127.20.231.129:43163)
0619 16:59:16.503440 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:16.503448 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 16:59:16.503516 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 16:59:16.503646 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 16:59:16.503646 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 16:59:16.503648 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 16:59:16.503729 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:16.503730 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:16.504371 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:16.504372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:16.505077 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 16:59:16.505079 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 16:59:16.505658 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 16:59:16.505659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 16:59:16.505734 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:16.505738 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 16:59:16.505814 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 16:59:16.505854 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":101,"thread_start_us":111,"threads_started":1}
I20260619 16:59:16.506354 32045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7319100c4d14a0ab4048e8330d24df4" candidate_uuid: "e29f9333926146a0bd240ed63dd20ef9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae748c996841450bb27312ae24e45e13" is_pre_election: true
I20260619 16:59:16.506531 32045 raft_consensus.cc:2454] T a7319100c4d14a0ab4048e8330d24df4 P ae748c996841450bb27312ae24e45e13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e29f9333926146a0bd240ed63dd20ef9 in term 0.
I20260619 16:59:16.506528 31911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7319100c4d14a0ab4048e8330d24df4" candidate_uuid: "e29f9333926146a0bd240ed63dd20ef9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "150f7cccc6444c1181216961c7577199" is_pre_election: true
I20260619 16:59:16.506698 31911 raft_consensus.cc:2454] T a7319100c4d14a0ab4048e8330d24df4 P 150f7cccc6444c1181216961c7577199 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e29f9333926146a0bd240ed63dd20ef9 in term 0.
I20260619 16:59:16.506762 31710 leader_election.cc:302] T a7319100c4d14a0ab4048e8330d24df4 P e29f9333926146a0bd240ed63dd20ef9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae748c996841450bb27312ae24e45e13, e29f9333926146a0bd240ed63dd20ef9; no voters:
I20260619 16:59:16.506916 32116 raft_consensus.cc:2781] T a7319100c4d14a0ab4048e8330d24df4 P e29f9333926146a0bd240ed63dd20ef9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 16:59:16.506971 32116 raft_consensus.cc:482] T a7319100c4d14a0ab4048e8330d24df4 P e29f9333926146a0bd240ed63dd20ef9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 16:59:16.507040 32116 raft_consensus.cc:3037] T a7319100c4d14a0ab4048e8330d24df4 P e29f9333926146a0bd240ed63dd20ef9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:59:16.507797 32116 raft_consensus.cc:504] T a7319100c4d14a0ab4048e8330d24df4 P e29f9333926146a0bd240ed63dd20ef9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae748c996841450bb27312ae24e45e13" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 38021 } } peers { permanent_uuid: "e29f9333926146a0bd240ed63dd20ef9" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 39305 } } peers { permanent_uuid: "150f7cccc6444c1181216961c7577199" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46529 } }
I20260619 16:59:16.507939 32116 leader_election.cc:288] T a7319100c4d14a0ab4048e8330d24df4 P e29f9333926146a0bd240ed63dd20ef9 [CANDIDATE]: Term 1 election: Requested vote from peers ae748c996841450bb27312ae24e45e13 (127.20.231.131:38021), 150f7cccc6444c1181216961c7577199 (127.20.231.130:46529)
I20260619 16:59:16.508091 31911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7319100c4d14a0ab4048e8330d24df4" candidate_uuid: "e29f9333926146a0bd240ed63dd20ef9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "150f7cccc6444c1181216961c7577199"
I20260619 16:59:16.508143 32045 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a7319100c4d14a0ab4048e8330d24df4" candidate_uuid: "e29f9333926146a0bd240ed63dd20ef9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae748c996841450bb27312ae24e45e13"
I20260619 16:59:16.508173 31911 raft_consensus.cc:3037] T a7319100c4d14a0ab4048e8330d24df4 P 150f7cccc6444c1181216961c7577199 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:59:16.508234 32045 raft_consensus.cc:3037] T a7319100c4d14a0ab4048e8330d24df4 P ae748c996841450bb27312ae24e45e13 [term 0 FOLLOWER]: Advancing to term 1
I20260619 16:59:16.509088 31911 raft_consensus.cc:2454] T a7319100c4d14a0ab4048e8330d24df4 P 150f7cccc6444c1181216961c7577199 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e29f9333926146a0bd240ed63dd20ef9 in term 1.
I20260619 16:59:16.509088 32045 raft_consensus.cc:2454] T a7319100c4d14a0ab4048e8330d24df4 P ae748c996841450bb27312ae24e45e13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e29f9333926146a0bd240ed63dd20ef9 in term 1.
I20260619 16:59:16.509282 31712 leader_election.cc:302] T a7319100c4d14a0ab4048e8330d24df4 P e29f9333926146a0bd240ed63dd20ef9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 150f7cccc6444c1181216961c7577199, e29f9333926146a0bd240ed63dd20ef9; no voters:
I20260619 16:59:16.509400 32116 raft_consensus.cc:2781] T a7319100c4d14a0ab4048e8330d24df4 P e29f9333926146a0bd240ed63dd20ef9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 16:59:16.509531 32116 raft_consensus.cc:686] T a7319100c4d14a0ab4048e8330d24df4 P e29f9333926146a0bd240ed63dd20ef9 [term 1 LEADER]: Becoming Leader. State: Replica: e29f9333926146a0bd240ed63dd20ef9, State: Running, Role: LEADER
I20260619 16:59:16.509624 32116 consensus_queue.cc:237] T a7319100c4d14a0ab4048e8330d24df4 P e29f9333926146a0bd240ed63dd20ef9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae748c996841450bb27312ae24e45e13" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 38021 } } peers { permanent_uuid: "e29f9333926146a0bd240ed63dd20ef9" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 39305 } } peers { permanent_uuid: "150f7cccc6444c1181216961c7577199" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46529 } }
I20260619 16:59:16.510581 31631 catalog_manager.cc:5495] T a7319100c4d14a0ab4048e8330d24df4 P e29f9333926146a0bd240ed63dd20ef9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e29f9333926146a0bd240ed63dd20ef9 (127.20.231.129). New cstate: current_term: 1 leader_uuid: "e29f9333926146a0bd240ed63dd20ef9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ae748c996841450bb27312ae24e45e13" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 38021 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e29f9333926146a0bd240ed63dd20ef9" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 39305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "150f7cccc6444c1181216961c7577199" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46529 } health_report { overall_health: UNKNOWN } } }
W20260619 16:59:16.523748 31824 tablet.cc:2369] T a7319100c4d14a0ab4048e8330d24df4 P e29f9333926146a0bd240ed63dd20ef9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 16:59:16.544229 32104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:16.541466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37328 (local address 127.20.231.129:39305)
0619 16:59:16.541566 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0619 16:59:16.541568 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 16:59:16.541577 (+ 9us) server_negotiation.cc:407] Connection header received
0619 16:59:16.541614 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 16:59:16.541615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 16:59:16.541634 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 16:59:16.541712 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 16:59:16.541945 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:16.542451 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:16.543278 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 16:59:16.543485 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 16:59:16.543546 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 16:59:16.543819 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 16:59:16.543929 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 16:59:16.543993 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 16:59:16.544050 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 16:59:16.544073 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 16:59:16.546164 32045 raft_consensus.cc:1261] T a7319100c4d14a0ab4048e8330d24df4 P ae748c996841450bb27312ae24e45e13 [term 1 FOLLOWER]: Refusing update from remote peer e29f9333926146a0bd240ed63dd20ef9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:59:16.546232 31911 raft_consensus.cc:1261] T a7319100c4d14a0ab4048e8330d24df4 P 150f7cccc6444c1181216961c7577199 [term 1 FOLLOWER]: Refusing update from remote peer e29f9333926146a0bd240ed63dd20ef9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 16:59:16.546618 32116 consensus_queue.cc:1035] T a7319100c4d14a0ab4048e8330d24df4 P e29f9333926146a0bd240ed63dd20ef9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae748c996841450bb27312ae24e45e13" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 38021 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:59:16.546784 32118 consensus_queue.cc:1035] T a7319100c4d14a0ab4048e8330d24df4 P e29f9333926146a0bd240ed63dd20ef9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "150f7cccc6444c1181216961c7577199" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 16:59:16.549705 32123 mvcc.cc:205] Tried to move back new op lower bound from 7298614708410007552 to 7298614708264017920. Current Snapshot: MvccSnapshot[applied={T|T < 7298614708410007552}]
I20260619 16:59:16.552124 31631 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:59:16.552295 31631 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43192:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 16:59:16.553184 31631 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5a257ea494ea4bbfa1f2128cb9f97012: Sending DeleteTablet for 3 replicas of tablet a7319100c4d14a0ab4048e8330d24df4
I20260619 16:59:16.553540 32025 tablet_service.cc:1514] Processing DeleteTablet for tablet a7319100c4d14a0ab4048e8330d24df4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:59:16 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:46032
I20260619 16:59:16.553579 31891 tablet_service.cc:1514] Processing DeleteTablet for tablet a7319100c4d14a0ab4048e8330d24df4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:59:16 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:48064
I20260619 16:59:16.553820 32132 tablet_replica.cc:330] T a7319100c4d14a0ab4048e8330d24df4 P 150f7cccc6444c1181216961c7577199: stopping tablet replica
I20260619 16:59:16.553794 31757 tablet_service.cc:1514] Processing DeleteTablet for tablet a7319100c4d14a0ab4048e8330d24df4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 16:59:16 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:37322
I20260619 16:59:16.553943 32132 raft_consensus.cc:2229] T a7319100c4d14a0ab4048e8330d24df4 P 150f7cccc6444c1181216961c7577199 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:59:16.554057 32133 tablet_replica.cc:330] T a7319100c4d14a0ab4048e8330d24df4 P e29f9333926146a0bd240ed63dd20ef9: stopping tablet replica
I20260619 16:59:16.554066 32132 raft_consensus.cc:2258] T a7319100c4d14a0ab4048e8330d24df4 P 150f7cccc6444c1181216961c7577199 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:59:16.554199 32133 raft_consensus.cc:2229] T a7319100c4d14a0ab4048e8330d24df4 P e29f9333926146a0bd240ed63dd20ef9 [term 1 LEADER]: Raft consensus shutting down.
I20260619 16:59:16.554379 32133 raft_consensus.cc:2258] T a7319100c4d14a0ab4048e8330d24df4 P e29f9333926146a0bd240ed63dd20ef9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:59:16.554502 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 31694
I20260619 16:59:16.554607 32132 ts_tablet_manager.cc:1905] T a7319100c4d14a0ab4048e8330d24df4 P 150f7cccc6444c1181216961c7577199: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:59:16.556530 32131 tablet_replica.cc:330] T a7319100c4d14a0ab4048e8330d24df4 P ae748c996841450bb27312ae24e45e13: stopping tablet replica
I20260619 16:59:16.556704 32131 raft_consensus.cc:2229] T a7319100c4d14a0ab4048e8330d24df4 P ae748c996841450bb27312ae24e45e13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 16:59:16.556797 32132 ts_tablet_manager.cc:1918] T a7319100c4d14a0ab4048e8330d24df4 P 150f7cccc6444c1181216961c7577199: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:59:16.556826 32131 raft_consensus.cc:2258] T a7319100c4d14a0ab4048e8330d24df4 P ae748c996841450bb27312ae24e45e13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 16:59:16.556879 32132 log.cc:1196] T a7319100c4d14a0ab4048e8330d24df4 P 150f7cccc6444c1181216961c7577199: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-1/wal/wals/a7319100c4d14a0ab4048e8330d24df4
I20260619 16:59:16.557322 32132 ts_tablet_manager.cc:1939] T a7319100c4d14a0ab4048e8330d24df4 P 150f7cccc6444c1181216961c7577199: Deleting consensus metadata
I20260619 16:59:16.557497 32131 ts_tablet_manager.cc:1905] T a7319100c4d14a0ab4048e8330d24df4 P ae748c996841450bb27312ae24e45e13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 16:59:16.557834 31618 catalog_manager.cc:4841] TS 150f7cccc6444c1181216961c7577199 (127.20.231.130:46529): tablet a7319100c4d14a0ab4048e8330d24df4 (table test-table [id=6868d8d1c60c4d69b64e464f29eab142]) successfully deleted
I20260619 16:59:16.559041 32131 ts_tablet_manager.cc:1918] T a7319100c4d14a0ab4048e8330d24df4 P ae748c996841450bb27312ae24e45e13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 16:59:16.559125 32131 log.cc:1196] T a7319100c4d14a0ab4048e8330d24df4 P ae748c996841450bb27312ae24e45e13: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781888301345144-21406-0/minicluster-data/ts-2/wal/wals/a7319100c4d14a0ab4048e8330d24df4
I20260619 16:59:16.559376 32131 ts_tablet_manager.cc:1939] T a7319100c4d14a0ab4048e8330d24df4 P ae748c996841450bb27312ae24e45e13: Deleting consensus metadata
I20260619 16:59:16.559818 31616 catalog_manager.cc:4841] TS ae748c996841450bb27312ae24e45e13 (127.20.231.131:38021): tablet a7319100c4d14a0ab4048e8330d24df4 (table test-table [id=6868d8d1c60c4d69b64e464f29eab142]) successfully deleted
W20260619 16:59:16.561045 31618 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.231.129:39305 (error 108)
I20260619 16:59:16.561264 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 31828
I20260619 16:59:16.562147 32103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 16:59:16.562005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:39305 (local address 127.0.0.1:37334)
0619 16:59:16.562061 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 16:59:16.562101 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.129:39305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 16:59:16.562280 31618 catalog_manager.cc:4584] TS e29f9333926146a0bd240ed63dd20ef9 (127.20.231.129:39305): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a7319100c4d14a0ab4048e8330d24df4: Network error: Client connection negotiation failed: client connection to 127.20.231.129:39305: connect: Connection refused (error 111)
I20260619 16:59:16.566748 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 31962
I20260619 16:59:16.571882 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 31600
2026-06-19T16:59:16Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2054 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 16:59:16 dist-test-slave-vnbj krb5kdc[32137](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 16:59:16 dist-test-slave-vnbj krb5kdc[32137](info): set up 2 sockets
Jun 19 16:59:16 dist-test-slave-vnbj krb5kdc[32137](info): commencing operation
krb5kdc: starting...
W20260619 16:59:17.655949 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 16:59:17 dist-test-slave-vnbj krb5kdc[32137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T16:59:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T16:59:17Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.20.231.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.20.231.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.20.231.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/rangeradmin_127.20.231.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.20.231.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/rangeradmin_127.20.231.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.20.231.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.20.231.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.20.231.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/rangerlookup_127.20.231.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.20.231.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/rangerlookup_127.20.231.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.231.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.231.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.231.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/HTTP_127.20.231.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.231.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/HTTP_127.20.231.148@KRBTEST.COM.keytab.
I20260619 16:59:17.835546 21406 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskGfxJfS/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/postgres -l logfile start
2026-06-19 16:59:20.511 UTC [32167] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 16:59:20.511 UTC [32167] LOG: listening on IPv4 address "127.20.231.148", port 58877
2026-06-19 16:59:20.515 UTC [32167] LOG: listening on Unix socket "/tmp/.s.PGSQL.58877"
2026-06-19 16:59:20.551 UTC [32170] LOG: database system was shut down at 2026-06-19 16:59:18 UTC
2026-06-19 16:59:20.556 UTC [32167] LOG: database system is ready to accept connections
I20260619 16:59:21.568607 21406 mini_postgres.cc:96] Postgres bound to 58877
2026-06-19 16:59:21.575 UTC [32178] FATAL: database "slave" does not exist
127.20.231.148:58877 - accepting connections
I20260619 16:59:21.575698 21406 mini_ranger.cc:161] Starting Ranger...
I20260619 16:59:21.591259 21406 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 16:59:22.117928 21406 mini_ranger.cc:87] Created ranger Postgres database
I20260619 16:59:22.118012 21406 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ranger-admin
2026-06-19 16:59:22,395 [I] DB FLAVOR :POSTGRES
2026-06-19 16:59:22,396 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 16:59:22,396 [I] Checking connection..
2026-06-19 16:59:22,396 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:58877/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 16:59:22,626 [I] Checking connection passed.
2026-06-19 16:59:22,626 [I] --------- Verifying version history table ---------
2026-06-19 16:59:22,626 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:58877/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 16:59:22,885 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 16:59:22,885 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:58877/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 16:59:23,140 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 16:59:23,140 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 16:59:23,140 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:58877/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 16:59:23.399 UTC [32242] WARNING: there is no transaction in progress
2026-06-19 16:59:23,406 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 16:59:23,406 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:58877/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 16:59:23,675 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 16:59:23,676 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 16:59:23,676 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:58877/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 16:59:23,909 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:58877/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 16:59:24,189 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:58877/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-vnbj.c.gcp-upstream.internal','N') ;"
2026-06-19 16:59:24,437 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:58877/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 16:59:24,679 [I] Table x_portal_user does not exist in database ranger
2026-06-19 16:59:24,679 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:58877/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 16:59:24,962 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 16:59:24,962 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:58877/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 16:59:25,225 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:58877/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 16:59:25,493 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 16:59:25,493 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:58877/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 16:59:26.490 UTC [32350] WARNING: there is no transaction in progress
2026-06-19 16:59:26.508 UTC [32350] WARNING: there is no transaction in progress
2026-06-19 16:59:26.525 UTC [32350] WARNING: there is no transaction in progress
2026-06-19 16:59:26.541 UTC [32350] WARNING: there is no transaction in progress
2026-06-19 16:59:26.559 UTC [32350] WARNING: there is no transaction in progress
2026-06-19 16:59:26.579 UTC [32350] WARNING: there is no transaction in progress
2026-06-19 16:59:26.640 UTC [32350] WARNING: there is no transaction in progress
2026-06-19 16:59:26.647 UTC [32350] WARNING: there is no transaction in progress
2026-06-19 16:59:26.662 UTC [32350] WARNING: there is no transaction in progress
2026-06-19 16:59:26.673 UTC [32350] WARNING: there is no transaction in progress
2026-06-19 16:59:26.683 UTC [32350] WARNING: there is no transaction in progress
2026-06-19 16:59:26.699 UTC [32350] WARNING: there is no transaction in progress
2026-06-19 16:59:26.710 UTC [32350] WARNING: there is no transaction in progress
2026-06-19 16:59:26.718 UTC [32350] WARNING: there is no transaction in progress
2026-06-19 16:59:26.726 UTC [32350] WARNING: there is no transaction in progress
2026-06-19 16:59:26.733 UTC [32350] WARNING: there is no transaction in progress
2026-06-19 16:59:27,415 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 16:59:27,416 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:58877/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 16:59:27,674 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 16:59:27,674 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:58877/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 16:59:27,933 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 16:59:27,933 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:58877/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 16:59:28,203 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 16:59:28,203 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:58877/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 16:59:28,474 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:58877/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 16:59:28,735 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:58877/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-vnbj.c.gcp-upstream.internal';"
2026-06-19 16:59:28,978 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 16:59:28,979 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:58877/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 16:59:29,225 [I] DB_PATCHES have already been applied
I20260619 16:59:29.230836 21406 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ranger-admin:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskGfxJfS/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 16:59:29.230980 21406 mini_ranger.cc:193] Using host: 127.20.231.148
I20260619 16:59:29.233946 21406 mini_ranger.cc:227] Ranger admin URL: http://127.20.231.148:45437
Jun 19, 2026 4:59:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 4:59:29 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 4:59:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 4:59:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 4:59:29 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 4:59:30 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 4:59:30 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 16:59:30.358827 21406 mini_ranger.cc:160] Time spent starting Ranger: real 8.783s user 0.000s sys 0.010s
Jun 19, 2026 4:59:51 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 4:59:51 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Jun 19, 2026 4:59:51 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 4:59:51 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 4:59:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 4:59:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 4:59:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 4:59:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 4:59:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 4:59:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 4:59:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 4:59:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 4:59:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 4:59:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 4:59:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 4:59:51 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 4:59:51 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 4:59:52 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 16:59:53.223594 21406 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.20.231.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.20.231.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.20.231.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/rangerkms_127.20.231.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.20.231.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/rangerkms_127.20.231.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.231.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.231.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.231.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/HTTP_127.20.231.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.231.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/HTTP_127.20.231.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 16:59:53 dist-test-slave-vnbj krb5kdc[32137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888393, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 16:59:53.312772 21406 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 16:59:53.322571 21406 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 16:59:54.269804 21406 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 16:59:54.269925 21406 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ranger-kms
I20260619 16:59:54.269949 21406 mini_ranger_kms.cc:227] Using postgres at 127.20.231.148:58877
2026-06-19 16:59:54,434 [I] DB FLAVOR :POSTGRES
2026-06-19 16:59:54,435 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 16:59:54,435 [I] Checking connection
2026-06-19 16:59:54,435 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:58877/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 16:59:54,670 [I] connection success
2026-06-19 16:59:54,670 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 16:59:54,670 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 16:59:54,670 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:58877/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 16:59:54,919 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 16:59:54,920 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 16:59:54,920 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 16:59:54,920 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:58877/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 16:59:55,198 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 16:59:55.719460 21406 mini_ranger_kms.cc:310] Created kms service
I20260619 16:59:55.895864 21406 mini_ranger_kms.cc:326] Created kudu user
I20260619 16:59:55.948750 21406 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 16:59:55 dist-test-slave-vnbj krb5kdc[32137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888393, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.20.231.148@KRBTEST.COM
I20260619 16:59:56.415583 21406 mini_ranger_kms.cc:384] Added ranger policy
I20260619 16:59:56.415697 21406 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ranger-kms:/tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 16:59:56.415719 21406 mini_ranger_kms.cc:240] Using host: 127.20.231.148
I20260619 16:59:56.419516 21406 mini_ranger_kms.cc:276] Ranger KMS PID: 32531
I20260619 16:59:56.419633 21406 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.20.231.148:35261
Jun 19, 2026 4:59:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 4:59:57 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 4:59:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 4:59:57 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 4:59:58 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 4:59:58 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260619 16:59:58.739125 21406 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.426s user 0.000s sys 0.144s
I20260619 16:59:58.739252 21406 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 16:59:58.739341 21406 mini_ranger_kms.cc:401] 127.20.231.148:35261/kms/v1/keys
Jun 19 16:59:58 dist-test-slave-vnbj krb5kdc[32137](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781888398, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.231.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 19 16:59:59 dist-test-slave-vnbj krb5kdc[32137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888398, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.231.148@KRBTEST.COM for HTTP/127.20.231.148@KRBTEST.COM
Jun 19 16:59:59 dist-test-slave-vnbj krb5kdc[32137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888398, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.231.148@KRBTEST.COM for HTTP/127.20.231.148@KRBTEST.COM
Jun 19 17:00:02 dist-test-slave-vnbj krb5kdc[32137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.20.231.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
I20260619 17:00:02.169894 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:37067
--webserver_interface=127.20.231.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:35369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:37067
--ranger_config_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.231.148:35261/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:00:02.252076 32581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:00:02.252350 32581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:00:02.252386 32581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:00:02.254238 32581 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:00:02.254336 32581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:00:02.254386 32581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:00:02.254419 32581 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:00:02.254469 32581 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:00:02.256260 32581 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:35369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/master-0/wal
--ranger_kms_url=127.20.231.148:35261/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:37067
--ranger_config_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:37067
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:00:02.256531 32581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:00:02.256771 32581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 17:00:02.262467 32581 server_base.cc:1034] running on GCE node
W20260619 17:00:02.262490 32586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:00:02.262476 32587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:00:02.262646 32589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:00:02.262996 32581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:00:02.263298 32581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:00:02.264464 32581 hybrid_clock.cc:648] HybridClock initialized: now 1781888402264450 us; error 32 us; skew 500 ppm
Jun 19 17:00:02 dist-test-slave-vnbj krb5kdc[32137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888402, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:00:02.272843 32581 init.cc:377] Logged in from keytab as kudu/127.20.231.190@KRBTEST.COM (short username kudu)
I20260619 17:00:02.273514 32581 webserver.cc:460] Webserver started at http://127.20.231.190:46483/ using document root <none> and password file <none>
I20260619 17:00:02.273779 32581 fs_manager.cc:359] Metadata directory not provided
I20260619 17:00:02.273834 32581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:00:02.273976 32581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 17:00:02 dist-test-slave-vnbj krb5kdc[32137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888402, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for HTTP/127.20.231.148@KRBTEST.COM
I20260619 17:00:02.319844 32581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "b471fdbf427b4215aa903a14c9cf8a63"
format_stamp: "Formatted at 2026-06-19 17:00:02 on dist-test-slave-vnbj"
server_key: "05407ba26c4758cb051bc2be790812e0"
server_key_iv: "d505bd9f41449a324ce740d3f2fb7c0d"
server_key_version: "kuduclusterkey@0"
I20260619 17:00:02.320264 32581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "b471fdbf427b4215aa903a14c9cf8a63"
format_stamp: "Formatted at 2026-06-19 17:00:02 on dist-test-slave-vnbj"
server_key: "05407ba26c4758cb051bc2be790812e0"
server_key_iv: "d505bd9f41449a324ce740d3f2fb7c0d"
server_key_version: "kuduclusterkey@0"
I20260619 17:00:02.321710 32581 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 17:00:02.336802 32596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:00:02.337255 32581 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:00:02.337369 32581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "b471fdbf427b4215aa903a14c9cf8a63"
format_stamp: "Formatted at 2026-06-19 17:00:02 on dist-test-slave-vnbj"
server_key: "05407ba26c4758cb051bc2be790812e0"
server_key_iv: "d505bd9f41449a324ce740d3f2fb7c0d"
server_key_version: "kuduclusterkey@0"
I20260619 17:00:02.337486 32581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:00:02.349094 32581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:00:02.351235 32581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:00:02.351411 32581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:00:02.356355 32581 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:37067
I20260619 17:00:02.356381 32648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:37067 every 8 connection(s)
I20260619 17:00:02.356812 32581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 17:00:02.357512 32649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:00:02.359881 32649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b471fdbf427b4215aa903a14c9cf8a63: Bootstrap starting.
I20260619 17:00:02.360476 32649 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b471fdbf427b4215aa903a14c9cf8a63: Neither blocks nor log segments found. Creating new log.
I20260619 17:00:02.360769 32649 log.cc:824] T 00000000000000000000000000000000 P b471fdbf427b4215aa903a14c9cf8a63: Log is configured to *not* fsync() on all Append() calls
I20260619 17:00:02.361441 32649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b471fdbf427b4215aa903a14c9cf8a63: No bootstrap required, opened a new log
I20260619 17:00:02.363715 32649 raft_consensus.cc:348] T 00000000000000000000000000000000 P b471fdbf427b4215aa903a14c9cf8a63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b471fdbf427b4215aa903a14c9cf8a63" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37067 } }
I20260619 17:00:02.363865 32649 raft_consensus.cc:374] T 00000000000000000000000000000000 P b471fdbf427b4215aa903a14c9cf8a63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:00:02.363890 32649 raft_consensus.cc:729] T 00000000000000000000000000000000 P b471fdbf427b4215aa903a14c9cf8a63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b471fdbf427b4215aa903a14c9cf8a63, State: Initialized, Role: FOLLOWER
I20260619 17:00:02.364045 32649 consensus_queue.cc:260] T 00000000000000000000000000000000 P b471fdbf427b4215aa903a14c9cf8a63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b471fdbf427b4215aa903a14c9cf8a63" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37067 } }
I20260619 17:00:02.364123 32649 raft_consensus.cc:388] T 00000000000000000000000000000000 P b471fdbf427b4215aa903a14c9cf8a63 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 17:00:02.364147 32649 raft_consensus.cc:482] T 00000000000000000000000000000000 P b471fdbf427b4215aa903a14c9cf8a63 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 17:00:02.364214 32649 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b471fdbf427b4215aa903a14c9cf8a63 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:00:02.364784 32649 raft_consensus.cc:504] T 00000000000000000000000000000000 P b471fdbf427b4215aa903a14c9cf8a63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b471fdbf427b4215aa903a14c9cf8a63" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37067 } }
I20260619 17:00:02.364871 32649 leader_election.cc:302] T 00000000000000000000000000000000 P b471fdbf427b4215aa903a14c9cf8a63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b471fdbf427b4215aa903a14c9cf8a63; no voters:
I20260619 17:00:02.365096 32649 leader_election.cc:288] T 00000000000000000000000000000000 P b471fdbf427b4215aa903a14c9cf8a63 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 17:00:02.365124 32652 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b471fdbf427b4215aa903a14c9cf8a63 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:00:02.365259 32652 raft_consensus.cc:686] T 00000000000000000000000000000000 P b471fdbf427b4215aa903a14c9cf8a63 [term 1 LEADER]: Becoming Leader. State: Replica: b471fdbf427b4215aa903a14c9cf8a63, State: Running, Role: LEADER
I20260619 17:00:02.365371 32649 sys_catalog.cc:564] T 00000000000000000000000000000000 P b471fdbf427b4215aa903a14c9cf8a63 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:00:02.365407 32652 consensus_queue.cc:237] T 00000000000000000000000000000000 P b471fdbf427b4215aa903a14c9cf8a63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b471fdbf427b4215aa903a14c9cf8a63" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37067 } }
I20260619 17:00:02.365818 32652 sys_catalog.cc:455] T 00000000000000000000000000000000 P b471fdbf427b4215aa903a14c9cf8a63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b471fdbf427b4215aa903a14c9cf8a63. Latest consensus state: current_term: 1 leader_uuid: "b471fdbf427b4215aa903a14c9cf8a63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b471fdbf427b4215aa903a14c9cf8a63" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37067 } } }
I20260619 17:00:02.365955 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 32581
I20260619 17:00:02.365994 32652 sys_catalog.cc:458] T 00000000000000000000000000000000 P b471fdbf427b4215aa903a14c9cf8a63 [sys.catalog]: This master's current role is: LEADER
I20260619 17:00:02.366055 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 17:00:02.366194 32661 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 17:00:02.366247 21406 external_mini_cluster.cc:1428] Setting key 2f6a5188466d72e12f31e894532238ca
I20260619 17:00:02.366698 32661 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 17:00:02.367048 32649 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 17:00:02.367436 32654 sys_catalog.cc:455] T 00000000000000000000000000000000 P b471fdbf427b4215aa903a14c9cf8a63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b471fdbf427b4215aa903a14c9cf8a63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b471fdbf427b4215aa903a14c9cf8a63" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37067 } } }
I20260619 17:00:02.367597 32654 sys_catalog.cc:458] T 00000000000000000000000000000000 P b471fdbf427b4215aa903a14c9cf8a63 [sys.catalog]: This master's current role is: LEADER
I20260619 17:00:02.371196 32670 mvcc.cc:205] Tried to move back new op lower bound from 7298614896097890304 to 7298614896089448448. Current Snapshot: MvccSnapshot[applied={T|T < 7298614896097890304}]
Jun 19 17:00:02 dist-test-slave-vnbj krb5kdc[32137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:00:02.379753 32661 catalog_manager.cc:1347] Generated new cluster ID: 4dafd722ffc24af3bdbb8a2448953ee9
I20260619 17:00:02.379817 32661 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 17:00:02.388989 32661 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 17:00:02.389832 32661 catalog_manager.cc:1504] Loading token signing keys...
I20260619 17:00:02.400985 32669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:02.366907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37692 (local address 127.20.231.190:37067)
0619 17:00:02.371119 (+ 4212us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:02.371153 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:02.371210 (+ 57us) server_negotiation.cc:407] Connection header received
0619 17:00:02.371463 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:02.371475 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:02.371555 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:02.371724 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:00:02.372638 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:02.374234 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:02.375490 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:02.376667 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:02.390053 (+ 13386us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:02.390161 (+ 108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:00:02.390164 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:00:02.390164 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:00:02.390174 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:00:02.398368 (+ 8194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:02.398905 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:02.398910 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:02.398910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:02.399021 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:02.399246 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:02.399247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:02.399247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:02.399561 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:00:02.400466 (+ 905us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:02.400548 (+ 82us) server_negotiation.cc:299] Negotiation successful
0619 17:00:02.400709 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4078,"thread_start_us":81,"threads_started":1}
I20260619 17:00:02.402120 32661 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b471fdbf427b4215aa903a14c9cf8a63: Generated new TSK 0
I20260619 17:00:02.402359 32661 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.231.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab.
Entry for principal kudu/127.20.231.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab.
I20260619 17:00:03.125640 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.231.148:35261/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.129:0
--local_ip_for_outbound_sockets=127.20.231.129
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:37067
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:35369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:00:03.215776 32694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:00:03.216015 32694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:00:03.216055 32694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:00:03.217264 32694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:00:03.217331 32694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:00:03.217374 32694 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.129
I20260619 17:00:03.218991 32694 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:35369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.20.231.148:35261/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:37067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.129
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:00:03.219259 32694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:00:03.219492 32694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:00:03.225078 32700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:00:03.225135 32702 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:00:03.225299 32694 server_base.cc:1034] running on GCE node
W20260619 17:00:03.225366 32699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:00:03.225647 32694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:00:03.225960 32694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:00:03.227135 32694 hybrid_clock.cc:648] HybridClock initialized: now 1781888403227099 us; error 55 us; skew 500 ppm
Jun 19 17:00:03 dist-test-slave-vnbj krb5kdc[32137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888403, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:00:03.230347 32694 init.cc:377] Logged in from keytab as kudu/127.20.231.129@KRBTEST.COM (short username kudu)
I20260619 17:00:03.230813 32694 webserver.cc:460] Webserver started at http://127.20.231.129:35055/ using document root <none> and password file <none>
I20260619 17:00:03.231127 32694 fs_manager.cc:359] Metadata directory not provided
I20260619 17:00:03.231204 32694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:00:03.231350 32694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 17:00:03 dist-test-slave-vnbj krb5kdc[32137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888403, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for HTTP/127.20.231.148@KRBTEST.COM
I20260619 17:00:03.245285 32694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-0/data/instance:
uuid: "3a942d9b1c76420db11f4729624c6d3e"
format_stamp: "Formatted at 2026-06-19 17:00:03 on dist-test-slave-vnbj"
server_key: "a7249350d185cb5b9a0e5cc2baa2b7d6"
server_key_iv: "7a55ef1fb6b8e0263f1e400f4772a878"
server_key_version: "kuduclusterkey@0"
I20260619 17:00:03.245682 32694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance:
uuid: "3a942d9b1c76420db11f4729624c6d3e"
format_stamp: "Formatted at 2026-06-19 17:00:03 on dist-test-slave-vnbj"
server_key: "a7249350d185cb5b9a0e5cc2baa2b7d6"
server_key_iv: "7a55ef1fb6b8e0263f1e400f4772a878"
server_key_version: "kuduclusterkey@0"
I20260619 17:00:03.247257 32694 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 17:00:03.261098 32709 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:00:03.261432 32694 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 17:00:03.261549 32694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-0/wal
uuid: "3a942d9b1c76420db11f4729624c6d3e"
format_stamp: "Formatted at 2026-06-19 17:00:03 on dist-test-slave-vnbj"
server_key: "a7249350d185cb5b9a0e5cc2baa2b7d6"
server_key_iv: "7a55ef1fb6b8e0263f1e400f4772a878"
server_key_version: "kuduclusterkey@0"
I20260619 17:00:03.261662 32694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:00:03.284466 32694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:00:03.287007 32694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:00:03.287222 32694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:00:03.287523 32694 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:00:03.287947 32694 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:00:03.288025 32694 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:00:03.288086 32694 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:00:03.288131 32694 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:00:03.295794 32694 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.129:43353
I20260619 17:00:03.296322 32694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
I20260619 17:00:03.297082 355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.129:43353 every 8 connection(s)
I20260619 17:00:03.302502 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 32694
I20260619 17:00:03.302616 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance
I20260619 17:00:03.302804 21406 external_mini_cluster.cc:1428] Setting key 8d0eb97afbafe171b02476e890889dfc
Jun 19 17:00:03 dist-test-slave-vnbj krb5kdc[32137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888403, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:00:03.315658 359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:03.297878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:37067 (local address 127.20.231.129:45205)
0619 17:00:03.298492 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:03.298540 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 17:00:03.299226 (+ 686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:00:03.299643 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:00:03.299648 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:00:03.299903 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:00:03.300179 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:03.300185 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:03.301216 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:03.301217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:03.302172 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:03.302174 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:03.303068 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:03.303069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:03.303441 (+ 372us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:03.304034 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:00:03.304045 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:00:03.311262 (+ 7217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:00:03.313637 (+ 2375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:00:03.313649 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:00:03.313653 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:00:03.314104 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:00:03.314352 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:00:03.314353 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:00:03.314353 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:00:03.314516 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:00:03.314895 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:00:03.314903 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:00:03.315107 (+ 204us) client_negotiation.cc:769] Sending connection context
0619 17:00:03.315222 (+ 115us) client_negotiation.cc:240] Negotiation successful
0619 17:00:03.315395 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":83,"threads_started":1}
I20260619 17:00:03.316270 360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:03.298173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:45205 (local address 127.20.231.190:37067)
0619 17:00:03.298331 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:03.298333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:03.298650 (+ 317us) server_negotiation.cc:407] Connection header received
0619 17:00:03.299396 (+ 746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:03.299397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:03.299412 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:03.299457 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:00:03.300296 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:03.301093 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:03.302285 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:03.302559 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:03.302995 (+ 436us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:03.311442 (+ 8447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:00:03.311444 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:00:03.311445 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:00:03.311456 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:00:03.313520 (+ 2064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:03.314204 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:03.314205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:03.314206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:03.314277 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:03.314596 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:03.314597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:03.314598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:03.314765 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:00:03.314839 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:03.316069 (+ 1230us) server_negotiation.cc:299] Negotiation successful
0619 17:00:03.316169 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":86,"threads_started":1}
WARNING: no policy specified for kudu/127.20.231.130@KRBTEST.COM; defaulting to no policy
I20260619 17:00:03.318153 356 heartbeater.cc:347] Connected to a master server at 127.20.231.190:37067
I20260619 17:00:03.318265 356 heartbeater.cc:464] Registering TS with master...
I20260619 17:00:03.318487 356 heartbeater.cc:511] Master 127.20.231.190:37067 requested a full tablet report, sending...
I20260619 17:00:03.319360 32613 ts_manager.cc:194] Registered new tserver with Master: 3a942d9b1c76420db11f4729624c6d3e (127.20.231.129:43353)
I20260619 17:00:03.320324 32613 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.129@KRBTEST.COM'} at 127.20.231.129:45205
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab.
Entry for principal kudu/127.20.231.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab.
I20260619 17:00:03.346256 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.231.148:35261/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.130:0
--local_ip_for_outbound_sockets=127.20.231.130
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:37067
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:35369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:00:03.470955 363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:00:03.471335 363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:00:03.471424 363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:00:03.473280 363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:00:03.473433 363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:00:03.473526 363 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.130
I20260619 17:00:03.475903 363 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:35369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.20.231.148:35261/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:37067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.130
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:00:03.476447 363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:00:03.476806 363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:00:03.484134 370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:00:03.484484 363 server_base.cc:1034] running on GCE node
W20260619 17:00:03.484269 373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:00:03.485404 371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:00:03.485742 363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:00:03.486052 363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:00:03.487239 363 hybrid_clock.cc:648] HybridClock initialized: now 1781888403487215 us; error 39 us; skew 500 ppm
Jun 19 17:00:03 dist-test-slave-vnbj krb5kdc[32137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888403, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:00:03.491040 363 init.cc:377] Logged in from keytab as kudu/127.20.231.130@KRBTEST.COM (short username kudu)
I20260619 17:00:03.491600 363 webserver.cc:460] Webserver started at http://127.20.231.130:45981/ using document root <none> and password file <none>
I20260619 17:00:03.491881 363 fs_manager.cc:359] Metadata directory not provided
I20260619 17:00:03.491940 363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:00:03.492087 363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 17:00:03 dist-test-slave-vnbj krb5kdc[32137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888403, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for HTTP/127.20.231.148@KRBTEST.COM
I20260619 17:00:03.509804 363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-1/data/instance:
uuid: "8f496cf075094ffaa978f6935f9f5d03"
format_stamp: "Formatted at 2026-06-19 17:00:03 on dist-test-slave-vnbj"
server_key: "5deb8c768591af19ba026a150aa2bf01"
server_key_iv: "05922aa990a2c4ec3d60b62a416e1144"
server_key_version: "kuduclusterkey@0"
I20260619 17:00:03.510442 363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance:
uuid: "8f496cf075094ffaa978f6935f9f5d03"
format_stamp: "Formatted at 2026-06-19 17:00:03 on dist-test-slave-vnbj"
server_key: "5deb8c768591af19ba026a150aa2bf01"
server_key_iv: "05922aa990a2c4ec3d60b62a416e1144"
server_key_version: "kuduclusterkey@0"
I20260619 17:00:03.514221 363 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.002s
I20260619 17:00:03.528411 380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:00:03.528661 363 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 17:00:03.528745 363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-1/wal
uuid: "8f496cf075094ffaa978f6935f9f5d03"
format_stamp: "Formatted at 2026-06-19 17:00:03 on dist-test-slave-vnbj"
server_key: "5deb8c768591af19ba026a150aa2bf01"
server_key_iv: "05922aa990a2c4ec3d60b62a416e1144"
server_key_version: "kuduclusterkey@0"
I20260619 17:00:03.528838 363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Jun 19 17:00:03 dist-test-slave-vnbj krb5kdc[32137](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781888403, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:00:03.568339 363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:00:03.571185 363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:00:03.571453 363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:00:03.571740 363 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:00:03.572188 363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:00:03.572294 363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:00:03.572371 363 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:00:03.572441 363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:00:03.581538 363 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.130:35671
I20260619 17:00:03.582123 363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
I20260619 17:00:03.582690 493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.130:35671 every 8 connection(s)
Jun 19 17:00:03 dist-test-slave-vnbj krb5kdc[32137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888403, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:00:03.592235 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 363
I20260619 17:00:03.592342 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance
I20260619 17:00:03.592540 21406 external_mini_cluster.cc:1428] Setting key 77c1a65cafbb85339028403f2088952b
I20260619 17:00:03.595502 360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:03.583620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:33941 (local address 127.20.231.190:37067)
0619 17:00:03.583712 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:03.583715 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:03.584064 (+ 349us) server_negotiation.cc:407] Connection header received
0619 17:00:03.584780 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:03.584782 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:03.584799 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:03.584830 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:00:03.585715 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:03.586517 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:03.588154 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:03.588470 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:03.588516 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:03.591350 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:00:03.591352 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:00:03.591353 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:00:03.591361 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:00:03.593199 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:03.594257 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:03.594259 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:03.594259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:03.594327 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:03.594640 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:03.594641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:03.594642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:03.594812 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:00:03.594879 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:03.595297 (+ 418us) server_negotiation.cc:299] Negotiation successful
0619 17:00:03.595399 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 17:00:03.596098 496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:03.583326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:37067 (local address 127.20.231.130:33941)
0619 17:00:03.583911 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:03.583954 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 17:00:03.584639 (+ 685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:00:03.585028 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:00:03.585033 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:00:03.585295 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:00:03.585585 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:03.585592 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:03.586760 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:03.586761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:03.588046 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:03.588050 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:03.588617 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:03.588618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:03.588775 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:03.589551 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:00:03.589562 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:00:03.591240 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:00:03.593824 (+ 2584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:00:03.593832 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:00:03.593835 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:00:03.594109 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:00:03.594412 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:00:03.594413 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:00:03.594414 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:00:03.594561 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:00:03.594930 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:00:03.594936 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:00:03.595114 (+ 178us) client_negotiation.cc:769] Sending connection context
0619 17:00:03.595641 (+ 527us) client_negotiation.cc:240] Negotiation successful
0619 17:00:03.595826 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":92,"threads_started":1}
I20260619 17:00:03.596783 494 heartbeater.cc:347] Connected to a master server at 127.20.231.190:37067
I20260619 17:00:03.596879 494 heartbeater.cc:464] Registering TS with master...
I20260619 17:00:03.597070 494 heartbeater.cc:511] Master 127.20.231.190:37067 requested a full tablet report, sending...
I20260619 17:00:03.597421 32613 ts_manager.cc:194] Registered new tserver with Master: 8f496cf075094ffaa978f6935f9f5d03 (127.20.231.130:35671)
I20260619 17:00:03.597894 32613 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.130@KRBTEST.COM'} at 127.20.231.130:33941
WARNING: no policy specified for kudu/127.20.231.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab.
Entry for principal kudu/127.20.231.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab.
I20260619 17:00:03.630537 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.231.148:35261/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.131:0
--local_ip_for_outbound_sockets=127.20.231.131
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:37067
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:35369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:00:03.758126 499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:00:03.758502 499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:00:03.758595 499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:00:03.760461 499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:00:03.760684 499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:00:03.760779 499 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.131
I20260619 17:00:03.763055 499 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:35369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.20.231.148:35261/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:37067
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.131
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:00:03.763567 499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:00:03.763893 499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:00:03.770814 506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:00:03.770977 505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:00:03.772017 499 server_base.cc:1034] running on GCE node
W20260619 17:00:03.770815 508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:00:03.772535 499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:00:03.772811 499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:00:03.773998 499 hybrid_clock.cc:648] HybridClock initialized: now 1781888403773971 us; error 37 us; skew 500 ppm
Jun 19 17:00:03 dist-test-slave-vnbj krb5kdc[32137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888403, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:00:03.777544 499 init.cc:377] Logged in from keytab as kudu/127.20.231.131@KRBTEST.COM (short username kudu)
I20260619 17:00:03.778105 499 webserver.cc:460] Webserver started at http://127.20.231.131:46569/ using document root <none> and password file <none>
I20260619 17:00:03.778379 499 fs_manager.cc:359] Metadata directory not provided
I20260619 17:00:03.778431 499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:00:03.778558 499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 17:00:03 dist-test-slave-vnbj krb5kdc[32137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888403, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for HTTP/127.20.231.148@KRBTEST.COM
I20260619 17:00:03.798744 499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-2/data/instance:
uuid: "ed97bccc64294756ad1468d0472888c7"
format_stamp: "Formatted at 2026-06-19 17:00:03 on dist-test-slave-vnbj"
server_key: "eb03139055454517273a658b9c14d7fc"
server_key_iv: "fd65e61470d1ab89227dc53beee16216"
server_key_version: "kuduclusterkey@0"
I20260619 17:00:03.799247 499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance:
uuid: "ed97bccc64294756ad1468d0472888c7"
format_stamp: "Formatted at 2026-06-19 17:00:03 on dist-test-slave-vnbj"
server_key: "eb03139055454517273a658b9c14d7fc"
server_key_iv: "fd65e61470d1ab89227dc53beee16216"
server_key_version: "kuduclusterkey@0"
I20260619 17:00:03.802692 499 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260619 17:00:03.825197 515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:00:03.825474 499 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260619 17:00:03.825582 499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-2/wal
uuid: "ed97bccc64294756ad1468d0472888c7"
format_stamp: "Formatted at 2026-06-19 17:00:03 on dist-test-slave-vnbj"
server_key: "eb03139055454517273a658b9c14d7fc"
server_key_iv: "fd65e61470d1ab89227dc53beee16216"
server_key_version: "kuduclusterkey@0"
I20260619 17:00:03.825685 499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:00:03.839408 499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:00:03.842056 499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:00:03.842270 499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:00:03.842531 499 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:00:03.842988 499 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:00:03.843083 499 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:00:03.843158 499 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:00:03.843216 499 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:00:03.849637 499 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.131:41227
I20260619 17:00:03.850150 499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
I20260619 17:00:03.851269 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 499
I20260619 17:00:03.851367 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance
I20260619 17:00:03.851564 21406 external_mini_cluster.cc:1428] Setting key c12939ba7f6f6f3d0d104fa1b63efdd6
I20260619 17:00:03.854614 628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.131:41227 every 8 connection(s)
Jun 19 17:00:03 dist-test-slave-vnbj krb5kdc[32137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888403, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:00:03.870407 360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:03.855283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:36429 (local address 127.20.231.190:37067)
0619 17:00:03.855328 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:03.855331 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:03.859115 (+ 3784us) server_negotiation.cc:407] Connection header received
0619 17:00:03.859884 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:03.859886 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:03.859899 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:03.859981 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:00:03.860724 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:03.861496 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:03.863342 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:03.863654 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:03.863698 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:03.866759 (+ 3061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:00:03.866761 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:00:03.866762 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:00:03.866771 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:00:03.868569 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:03.869144 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:03.869146 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:03.869146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:03.869209 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:03.869554 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:03.869555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:03.869556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:03.869719 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:00:03.869795 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:03.870196 (+ 401us) server_negotiation.cc:299] Negotiation successful
0619 17:00:03.870298 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 17:00:03.870968 631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:03.855294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:37067 (local address 127.20.231.131:36429)
0619 17:00:03.855505 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:03.858998 (+ 3493us) client_negotiation.cc:174] Beginning negotiation
0619 17:00:03.859756 (+ 758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:00:03.860077 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:00:03.860083 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:00:03.860333 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:00:03.860613 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:03.860619 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:03.861612 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:03.861613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:03.863165 (+ 1552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:03.863167 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:03.863791 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:03.863792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:03.863963 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:03.864632 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:00:03.864643 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:00:03.866646 (+ 2003us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:00:03.868672 (+ 2026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:00:03.868680 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:00:03.868683 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:00:03.869030 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:00:03.869287 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:00:03.869288 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:00:03.869288 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:00:03.869450 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:00:03.869826 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:00:03.869834 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:00:03.870014 (+ 180us) client_negotiation.cc:769] Sending connection context
0619 17:00:03.870526 (+ 512us) client_negotiation.cc:240] Negotiation successful
0619 17:00:03.870699 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":133,"threads_started":1}
I20260619 17:00:03.871642 629 heartbeater.cc:347] Connected to a master server at 127.20.231.190:37067
I20260619 17:00:03.871755 629 heartbeater.cc:464] Registering TS with master...
I20260619 17:00:03.871968 629 heartbeater.cc:511] Master 127.20.231.190:37067 requested a full tablet report, sending...
I20260619 17:00:03.872400 32613 ts_manager.cc:194] Registered new tserver with Master: ed97bccc64294756ad1468d0472888c7 (127.20.231.131:41227)
I20260619 17:00:03.872920 32613 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.131@KRBTEST.COM'} at 127.20.231.131:36429
I20260619 17:00:03.875535 21406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 17:00:03.885983 360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:03.879044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37706 (local address 127.20.231.190:37067)
0619 17:00:03.879104 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:03.879112 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:03.879309 (+ 197us) server_negotiation.cc:407] Connection header received
0619 17:00:03.879399 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:03.879399 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:03.879410 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:03.879456 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:00:03.880120 (+ 664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:03.880620 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:03.881452 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:03.881650 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:03.881750 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:03.883132 (+ 1382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:00:03.883134 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:00:03.883134 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:00:03.883141 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:00:03.884616 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:03.885091 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:03.885092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:03.885093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:03.885154 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:03.885375 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:03.885375 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:03.885376 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:03.885552 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:00:03.885645 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:03.885770 (+ 125us) server_negotiation.cc:299] Negotiation successful
0619 17:00:03.885859 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 17:00:03.887928 32613 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37706:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 17:00:03.889907 32613 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 17:00:03.897291 641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:03.892671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:41227 (local address 127.0.0.1:50760)
0619 17:00:03.893018 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:03.893067 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 17:00:03.893142 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:00:03.893429 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:00:03.893432 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:00:03.893434 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:00:03.893513 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:03.893515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:03.894944 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:03.894945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:03.895994 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:03.895996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:03.896991 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:03.896991 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:03.897079 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:03.897098 (+ 19us) client_negotiation.cc:769] Sending connection context
0619 17:00:03.897148 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 17:00:03.897200 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":21,"thread_start_us":409,"threads_started":1}
I20260619 17:00:03.897786 642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:03.892768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50760 (local address 127.20.231.131:41227)
0619 17:00:03.893236 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:03.893241 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:03.893253 (+ 12us) server_negotiation.cc:407] Connection header received
0619 17:00:03.893300 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:03.893302 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:03.893320 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:03.893591 (+ 271us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:00:03.893660 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:03.894810 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:03.896113 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:03.896895 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:03.897544 (+ 649us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:03.897617 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:03.897646 (+ 29us) server_negotiation.cc:299] Negotiation successful
0619 17:00:03.897692 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":102,"threads_started":1}
I20260619 17:00:03.897974 644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:03.892671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:35671 (local address 127.0.0.1:52318)
0619 17:00:03.893132 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:03.893143 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 17:00:03.893196 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:00:03.893870 (+ 674us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:00:03.893871 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:00:03.893873 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:00:03.893917 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:03.893919 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:03.895232 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:03.895233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:03.896272 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:03.896276 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:03.897714 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:03.897715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:03.897793 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:03.897799 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 17:00:03.897836 (+ 37us) client_negotiation.cc:240] Negotiation successful
0619 17:00:03.897884 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"mutex_wait_us":2,"thread_start_us":110,"threads_started":1}
I20260619 17:00:03.898538 646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:03.893025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52318 (local address 127.20.231.130:35671)
0619 17:00:03.893711 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:03.893715 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:03.893724 (+ 9us) server_negotiation.cc:407] Connection header received
0619 17:00:03.893760 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:03.893761 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:03.893775 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:03.893982 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:00:03.894045 (+ 63us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:03.895135 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:03.896404 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:03.897281 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:03.898329 (+ 1048us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:03.898390 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:03.898417 (+ 27us) server_negotiation.cc:299] Negotiation successful
0619 17:00:03.898461 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":653,"thread_start_us":95,"threads_started":1}
I20260619 17:00:03.898630 645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:03.892679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:43353 (local address 127.0.0.1:46790)
0619 17:00:03.893285 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:03.893293 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 17:00:03.893326 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:00:03.893590 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:00:03.893590 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:00:03.893592 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:00:03.893624 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:03.893625 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:03.894936 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:03.894938 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:03.897048 (+ 2110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:03.897050 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:03.898385 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:03.898386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:03.898457 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:03.898461 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 17:00:03.898518 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 17:00:03.898558 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"mutex_wait_us":329,"thread_start_us":333,"threads_started":1}
I20260619 17:00:03.898643 643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:03.892837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46790 (local address 127.20.231.129:43353)
0619 17:00:03.893425 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:03.893430 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:03.893457 (+ 27us) server_negotiation.cc:407] Connection header received
0619 17:00:03.893492 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:03.893494 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:03.893509 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:03.893582 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:00:03.893705 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:03.894824 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:03.897134 (+ 2310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:03.898299 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:03.898388 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:03.898441 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:03.898533 (+ 92us) server_negotiation.cc:299] Negotiation successful
0619 17:00:03.898572 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":72,"threads_started":1}
I20260619 17:00:03.898540 563 tablet_service.cc:1467] Processing CreateTablet for tablet a836ea5c1efe4a7f9f72300d5c616410 (DEFAULT_TABLE table=test-table [id=9ca2a4e523804d13ad04593ee599e85d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 17:00:03.898944 563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a836ea5c1efe4a7f9f72300d5c616410. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:00:03.899410 428 tablet_service.cc:1467] Processing CreateTablet for tablet a836ea5c1efe4a7f9f72300d5c616410 (DEFAULT_TABLE table=test-table [id=9ca2a4e523804d13ad04593ee599e85d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 17:00:03.899655 428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a836ea5c1efe4a7f9f72300d5c616410. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:00:03.899559 32757 tablet_service.cc:1467] Processing CreateTablet for tablet a836ea5c1efe4a7f9f72300d5c616410 (DEFAULT_TABLE table=test-table [id=9ca2a4e523804d13ad04593ee599e85d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 17:00:03.899865 32757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a836ea5c1efe4a7f9f72300d5c616410. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:00:03.902076 647 tablet_bootstrap.cc:492] T a836ea5c1efe4a7f9f72300d5c616410 P 8f496cf075094ffaa978f6935f9f5d03: Bootstrap starting.
I20260619 17:00:03.903045 649 tablet_bootstrap.cc:492] T a836ea5c1efe4a7f9f72300d5c616410 P 3a942d9b1c76420db11f4729624c6d3e: Bootstrap starting.
I20260619 17:00:03.903105 647 tablet_bootstrap.cc:654] T a836ea5c1efe4a7f9f72300d5c616410 P 8f496cf075094ffaa978f6935f9f5d03: Neither blocks nor log segments found. Creating new log.
I20260619 17:00:03.903399 647 log.cc:824] T a836ea5c1efe4a7f9f72300d5c616410 P 8f496cf075094ffaa978f6935f9f5d03: Log is configured to *not* fsync() on all Append() calls
I20260619 17:00:03.903797 649 tablet_bootstrap.cc:654] T a836ea5c1efe4a7f9f72300d5c616410 P 3a942d9b1c76420db11f4729624c6d3e: Neither blocks nor log segments found. Creating new log.
I20260619 17:00:03.904096 649 log.cc:824] T a836ea5c1efe4a7f9f72300d5c616410 P 3a942d9b1c76420db11f4729624c6d3e: Log is configured to *not* fsync() on all Append() calls
I20260619 17:00:03.904286 647 tablet_bootstrap.cc:492] T a836ea5c1efe4a7f9f72300d5c616410 P 8f496cf075094ffaa978f6935f9f5d03: No bootstrap required, opened a new log
I20260619 17:00:03.904417 647 ts_tablet_manager.cc:1397] T a836ea5c1efe4a7f9f72300d5c616410 P 8f496cf075094ffaa978f6935f9f5d03: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 17:00:03.904810 649 tablet_bootstrap.cc:492] T a836ea5c1efe4a7f9f72300d5c616410 P 3a942d9b1c76420db11f4729624c6d3e: No bootstrap required, opened a new log
I20260619 17:00:03.904896 649 ts_tablet_manager.cc:1397] T a836ea5c1efe4a7f9f72300d5c616410 P 3a942d9b1c76420db11f4729624c6d3e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 17:00:03.907042 648 tablet_bootstrap.cc:492] T a836ea5c1efe4a7f9f72300d5c616410 P ed97bccc64294756ad1468d0472888c7: Bootstrap starting.
I20260619 17:00:03.906944 647 raft_consensus.cc:348] T a836ea5c1efe4a7f9f72300d5c616410 P 8f496cf075094ffaa978f6935f9f5d03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a942d9b1c76420db11f4729624c6d3e" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 43353 } } peers { permanent_uuid: "ed97bccc64294756ad1468d0472888c7" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 41227 } } peers { permanent_uuid: "8f496cf075094ffaa978f6935f9f5d03" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 35671 } }
I20260619 17:00:03.907135 647 raft_consensus.cc:374] T a836ea5c1efe4a7f9f72300d5c616410 P 8f496cf075094ffaa978f6935f9f5d03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:00:03.907212 647 raft_consensus.cc:729] T a836ea5c1efe4a7f9f72300d5c616410 P 8f496cf075094ffaa978f6935f9f5d03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f496cf075094ffaa978f6935f9f5d03, State: Initialized, Role: FOLLOWER
I20260619 17:00:03.907151 649 raft_consensus.cc:348] T a836ea5c1efe4a7f9f72300d5c616410 P 3a942d9b1c76420db11f4729624c6d3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a942d9b1c76420db11f4729624c6d3e" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 43353 } } peers { permanent_uuid: "ed97bccc64294756ad1468d0472888c7" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 41227 } } peers { permanent_uuid: "8f496cf075094ffaa978f6935f9f5d03" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 35671 } }
I20260619 17:00:03.907327 649 raft_consensus.cc:374] T a836ea5c1efe4a7f9f72300d5c616410 P 3a942d9b1c76420db11f4729624c6d3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:00:03.907403 649 raft_consensus.cc:729] T a836ea5c1efe4a7f9f72300d5c616410 P 3a942d9b1c76420db11f4729624c6d3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a942d9b1c76420db11f4729624c6d3e, State: Initialized, Role: FOLLOWER
I20260619 17:00:03.907498 647 consensus_queue.cc:260] T a836ea5c1efe4a7f9f72300d5c616410 P 8f496cf075094ffaa978f6935f9f5d03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a942d9b1c76420db11f4729624c6d3e" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 43353 } } peers { permanent_uuid: "ed97bccc64294756ad1468d0472888c7" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 41227 } } peers { permanent_uuid: "8f496cf075094ffaa978f6935f9f5d03" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 35671 } }
I20260619 17:00:03.907622 649 consensus_queue.cc:260] T a836ea5c1efe4a7f9f72300d5c616410 P 3a942d9b1c76420db11f4729624c6d3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a942d9b1c76420db11f4729624c6d3e" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 43353 } } peers { permanent_uuid: "ed97bccc64294756ad1468d0472888c7" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 41227 } } peers { permanent_uuid: "8f496cf075094ffaa978f6935f9f5d03" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 35671 } }
I20260619 17:00:03.907925 647 ts_tablet_manager.cc:1428] T a836ea5c1efe4a7f9f72300d5c616410 P 8f496cf075094ffaa978f6935f9f5d03: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 17:00:03.907958 649 ts_tablet_manager.cc:1428] T a836ea5c1efe4a7f9f72300d5c616410 P 3a942d9b1c76420db11f4729624c6d3e: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 17:00:03.908066 356 heartbeater.cc:503] Master 127.20.231.190:37067 was elected leader, sending a full tablet report...
I20260619 17:00:03.908119 494 heartbeater.cc:503] Master 127.20.231.190:37067 was elected leader, sending a full tablet report...
I20260619 17:00:03.908141 648 tablet_bootstrap.cc:654] T a836ea5c1efe4a7f9f72300d5c616410 P ed97bccc64294756ad1468d0472888c7: Neither blocks nor log segments found. Creating new log.
I20260619 17:00:03.908430 648 log.cc:824] T a836ea5c1efe4a7f9f72300d5c616410 P ed97bccc64294756ad1468d0472888c7: Log is configured to *not* fsync() on all Append() calls
I20260619 17:00:03.910687 648 tablet_bootstrap.cc:492] T a836ea5c1efe4a7f9f72300d5c616410 P ed97bccc64294756ad1468d0472888c7: No bootstrap required, opened a new log
I20260619 17:00:03.910804 648 ts_tablet_manager.cc:1397] T a836ea5c1efe4a7f9f72300d5c616410 P ed97bccc64294756ad1468d0472888c7: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260619 17:00:03.912709 648 raft_consensus.cc:348] T a836ea5c1efe4a7f9f72300d5c616410 P ed97bccc64294756ad1468d0472888c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a942d9b1c76420db11f4729624c6d3e" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 43353 } } peers { permanent_uuid: "ed97bccc64294756ad1468d0472888c7" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 41227 } } peers { permanent_uuid: "8f496cf075094ffaa978f6935f9f5d03" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 35671 } }
I20260619 17:00:03.912935 648 raft_consensus.cc:374] T a836ea5c1efe4a7f9f72300d5c616410 P ed97bccc64294756ad1468d0472888c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:00:03.913007 648 raft_consensus.cc:729] T a836ea5c1efe4a7f9f72300d5c616410 P ed97bccc64294756ad1468d0472888c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed97bccc64294756ad1468d0472888c7, State: Initialized, Role: FOLLOWER
I20260619 17:00:03.913192 648 consensus_queue.cc:260] T a836ea5c1efe4a7f9f72300d5c616410 P ed97bccc64294756ad1468d0472888c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a942d9b1c76420db11f4729624c6d3e" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 43353 } } peers { permanent_uuid: "ed97bccc64294756ad1468d0472888c7" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 41227 } } peers { permanent_uuid: "8f496cf075094ffaa978f6935f9f5d03" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 35671 } }
I20260619 17:00:03.913506 648 ts_tablet_manager.cc:1428] T a836ea5c1efe4a7f9f72300d5c616410 P ed97bccc64294756ad1468d0472888c7: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260619 17:00:03.913645 629 heartbeater.cc:503] Master 127.20.231.190:37067 was elected leader, sending a full tablet report...
Jun 19 17:00:04 dist-test-slave-vnbj krb5kdc[32137](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888403, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for HTTP/127.20.231.148@KRBTEST.COM
W20260619 17:00:04.048939 357 tablet.cc:2369] T a836ea5c1efe4a7f9f72300d5c616410 P 3a942d9b1c76420db11f4729624c6d3e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 17:00:04.072238 652 raft_consensus.cc:482] T a836ea5c1efe4a7f9f72300d5c616410 P 8f496cf075094ffaa978f6935f9f5d03 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 17:00:04.072371 652 raft_consensus.cc:504] T a836ea5c1efe4a7f9f72300d5c616410 P 8f496cf075094ffaa978f6935f9f5d03 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a942d9b1c76420db11f4729624c6d3e" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 43353 } } peers { permanent_uuid: "ed97bccc64294756ad1468d0472888c7" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 41227 } } peers { permanent_uuid: "8f496cf075094ffaa978f6935f9f5d03" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 35671 } }
I20260619 17:00:04.072850 652 leader_election.cc:288] T a836ea5c1efe4a7f9f72300d5c616410 P 8f496cf075094ffaa978f6935f9f5d03 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a942d9b1c76420db11f4729624c6d3e (127.20.231.129:43353), ed97bccc64294756ad1468d0472888c7 (127.20.231.131:41227)
I20260619 17:00:04.077247 496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:04.072985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:43353 (local address 127.20.231.130:48935)
0619 17:00:04.073084 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:04.073098 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 17:00:04.073138 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:00:04.073390 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:00:04.073391 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:00:04.073393 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:00:04.073462 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:04.073464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:04.074434 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:04.074435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:04.075655 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:04.075657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:04.076975 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:04.076976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:04.077052 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:04.077056 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 17:00:04.077107 (+ 51us) client_negotiation.cc:240] Negotiation successful
0619 17:00:04.077151 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79}
I20260619 17:00:04.077557 643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:04.073175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:48935 (local address 127.20.231.129:43353)
0619 17:00:04.073239 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:04.073241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:04.073249 (+ 8us) server_negotiation.cc:407] Connection header received
0619 17:00:04.073277 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:04.073278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:04.073288 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:04.073335 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:00:04.073545 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:04.074347 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:04.076177 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:04.076734 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:04.076797 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:04.076807 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:04.077432 (+ 625us) server_negotiation.cc:299] Negotiation successful
0619 17:00:04.077480 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 17:00:04.077549 642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:04.073112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:52263 (local address 127.20.231.131:41227)
0619 17:00:04.073179 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:04.073182 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:04.073790 (+ 608us) server_negotiation.cc:407] Connection header received
0619 17:00:04.073900 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:04.073901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:04.073909 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:04.073938 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:00:04.074135 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:04.074871 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:04.076609 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:04.077130 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:04.077170 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:04.077177 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:04.077443 (+ 266us) server_negotiation.cc:299] Negotiation successful
0619 17:00:04.077483 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 17:00:04.077822 656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:04.073037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:41227 (local address 127.20.231.130:52263)
0619 17:00:04.073724 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:04.073735 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 17:00:04.073819 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:00:04.073997 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:00:04.073998 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:00:04.073999 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:00:04.074072 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:04.074073 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:04.074989 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:04.074990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:04.076077 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:04.076080 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:04.077267 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:04.077268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:04.077353 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:04.077357 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 17:00:04.077638 (+ 281us) client_negotiation.cc:240] Negotiation successful
0619 17:00:04.077684 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":671,"thread_start_us":89,"threads_started":1}
I20260619 17:00:04.078052 304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a836ea5c1efe4a7f9f72300d5c616410" candidate_uuid: "8f496cf075094ffaa978f6935f9f5d03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a942d9b1c76420db11f4729624c6d3e" is_pre_election: true
I20260619 17:00:04.078240 304 raft_consensus.cc:2454] T a836ea5c1efe4a7f9f72300d5c616410 P 3a942d9b1c76420db11f4729624c6d3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f496cf075094ffaa978f6935f9f5d03 in term 0.
I20260619 17:00:04.078235 580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a836ea5c1efe4a7f9f72300d5c616410" candidate_uuid: "8f496cf075094ffaa978f6935f9f5d03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed97bccc64294756ad1468d0472888c7" is_pre_election: true
I20260619 17:00:04.078398 580 raft_consensus.cc:2454] T a836ea5c1efe4a7f9f72300d5c616410 P ed97bccc64294756ad1468d0472888c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8f496cf075094ffaa978f6935f9f5d03 in term 0.
I20260619 17:00:04.078670 383 leader_election.cc:302] T a836ea5c1efe4a7f9f72300d5c616410 P 8f496cf075094ffaa978f6935f9f5d03 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a942d9b1c76420db11f4729624c6d3e, 8f496cf075094ffaa978f6935f9f5d03; no voters:
I20260619 17:00:04.078811 652 raft_consensus.cc:2781] T a836ea5c1efe4a7f9f72300d5c616410 P 8f496cf075094ffaa978f6935f9f5d03 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 17:00:04.078878 652 raft_consensus.cc:482] T a836ea5c1efe4a7f9f72300d5c616410 P 8f496cf075094ffaa978f6935f9f5d03 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 17:00:04.078902 652 raft_consensus.cc:3037] T a836ea5c1efe4a7f9f72300d5c616410 P 8f496cf075094ffaa978f6935f9f5d03 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:00:04.079833 652 raft_consensus.cc:504] T a836ea5c1efe4a7f9f72300d5c616410 P 8f496cf075094ffaa978f6935f9f5d03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a942d9b1c76420db11f4729624c6d3e" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 43353 } } peers { permanent_uuid: "ed97bccc64294756ad1468d0472888c7" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 41227 } } peers { permanent_uuid: "8f496cf075094ffaa978f6935f9f5d03" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 35671 } }
I20260619 17:00:04.079993 652 leader_election.cc:288] T a836ea5c1efe4a7f9f72300d5c616410 P 8f496cf075094ffaa978f6935f9f5d03 [CANDIDATE]: Term 1 election: Requested vote from peers 3a942d9b1c76420db11f4729624c6d3e (127.20.231.129:43353), ed97bccc64294756ad1468d0472888c7 (127.20.231.131:41227)
I20260619 17:00:04.080168 580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a836ea5c1efe4a7f9f72300d5c616410" candidate_uuid: "8f496cf075094ffaa978f6935f9f5d03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed97bccc64294756ad1468d0472888c7"
I20260619 17:00:04.080250 580 raft_consensus.cc:3037] T a836ea5c1efe4a7f9f72300d5c616410 P ed97bccc64294756ad1468d0472888c7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:00:04.080361 304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a836ea5c1efe4a7f9f72300d5c616410" candidate_uuid: "8f496cf075094ffaa978f6935f9f5d03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a942d9b1c76420db11f4729624c6d3e"
I20260619 17:00:04.080443 304 raft_consensus.cc:3037] T a836ea5c1efe4a7f9f72300d5c616410 P 3a942d9b1c76420db11f4729624c6d3e [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:00:04.081202 580 raft_consensus.cc:2454] T a836ea5c1efe4a7f9f72300d5c616410 P ed97bccc64294756ad1468d0472888c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f496cf075094ffaa978f6935f9f5d03 in term 1.
I20260619 17:00:04.081306 304 raft_consensus.cc:2454] T a836ea5c1efe4a7f9f72300d5c616410 P 3a942d9b1c76420db11f4729624c6d3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8f496cf075094ffaa978f6935f9f5d03 in term 1.
I20260619 17:00:04.081425 384 leader_election.cc:302] T a836ea5c1efe4a7f9f72300d5c616410 P 8f496cf075094ffaa978f6935f9f5d03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8f496cf075094ffaa978f6935f9f5d03, ed97bccc64294756ad1468d0472888c7; no voters:
I20260619 17:00:04.081593 652 raft_consensus.cc:2781] T a836ea5c1efe4a7f9f72300d5c616410 P 8f496cf075094ffaa978f6935f9f5d03 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:00:04.081769 652 raft_consensus.cc:686] T a836ea5c1efe4a7f9f72300d5c616410 P 8f496cf075094ffaa978f6935f9f5d03 [term 1 LEADER]: Becoming Leader. State: Replica: 8f496cf075094ffaa978f6935f9f5d03, State: Running, Role: LEADER
I20260619 17:00:04.081898 652 consensus_queue.cc:237] T a836ea5c1efe4a7f9f72300d5c616410 P 8f496cf075094ffaa978f6935f9f5d03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a942d9b1c76420db11f4729624c6d3e" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 43353 } } peers { permanent_uuid: "ed97bccc64294756ad1468d0472888c7" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 41227 } } peers { permanent_uuid: "8f496cf075094ffaa978f6935f9f5d03" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 35671 } }
I20260619 17:00:04.082952 32612 catalog_manager.cc:5495] T a836ea5c1efe4a7f9f72300d5c616410 P 8f496cf075094ffaa978f6935f9f5d03 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8f496cf075094ffaa978f6935f9f5d03 (127.20.231.130). New cstate: current_term: 1 leader_uuid: "8f496cf075094ffaa978f6935f9f5d03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a942d9b1c76420db11f4729624c6d3e" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 43353 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ed97bccc64294756ad1468d0472888c7" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 41227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8f496cf075094ffaa978f6935f9f5d03" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 35671 } health_report { overall_health: HEALTHY } } }
W20260619 17:00:04.083438 495 tablet.cc:2369] T a836ea5c1efe4a7f9f72300d5c616410 P 8f496cf075094ffaa978f6935f9f5d03: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 17:00:04.105911 630 tablet.cc:2369] T a836ea5c1efe4a7f9f72300d5c616410 P ed97bccc64294756ad1468d0472888c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 17:00:04.126042 646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:04.123613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52324 (local address 127.20.231.130:35671)
0619 17:00:04.123691 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:04.123694 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:04.123701 (+ 7us) server_negotiation.cc:407] Connection header received
0619 17:00:04.123742 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:04.123742 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:04.123766 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:04.123854 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 17:00:04.123966 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:04.124457 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:04.125238 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:04.125456 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:04.125532 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:04.125700 (+ 168us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 17:00:04.125814 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 17:00:04.125911 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:04.125931 (+ 20us) server_negotiation.cc:299] Negotiation successful
0619 17:00:04.125950 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 17:00:04.127962 580 raft_consensus.cc:1261] T a836ea5c1efe4a7f9f72300d5c616410 P ed97bccc64294756ad1468d0472888c7 [term 1 FOLLOWER]: Refusing update from remote peer 8f496cf075094ffaa978f6935f9f5d03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 17:00:04.127962 304 raft_consensus.cc:1261] T a836ea5c1efe4a7f9f72300d5c616410 P 3a942d9b1c76420db11f4729624c6d3e [term 1 FOLLOWER]: Refusing update from remote peer 8f496cf075094ffaa978f6935f9f5d03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 17:00:04.128392 657 consensus_queue.cc:1035] T a836ea5c1efe4a7f9f72300d5c616410 P 8f496cf075094ffaa978f6935f9f5d03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed97bccc64294756ad1468d0472888c7" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 41227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 17:00:04.128547 652 consensus_queue.cc:1035] T a836ea5c1efe4a7f9f72300d5c616410 P 8f496cf075094ffaa978f6935f9f5d03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a942d9b1c76420db11f4729624c6d3e" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 43353 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 17:00:04.134204 32613 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 17:00:04.134332 32613 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 17:00:04.135164 32613 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b471fdbf427b4215aa903a14c9cf8a63: Sending DeleteTablet for 3 replicas of tablet a836ea5c1efe4a7f9f72300d5c616410
I20260619 17:00:04.135634 662 mvcc.cc:205] Tried to move back new op lower bound from 7298614903305490432 to 7298614903120224256. Current Snapshot: MvccSnapshot[applied={T|T < 7298614903305490432}]
I20260619 17:00:04.135902 428 tablet_service.cc:1514] Processing DeleteTablet for tablet a836ea5c1efe4a7f9f72300d5c616410 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:00:04 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:52318
I20260619 17:00:04.135959 32757 tablet_service.cc:1514] Processing DeleteTablet for tablet a836ea5c1efe4a7f9f72300d5c616410 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:00:04 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:46790
I20260619 17:00:04.136215 670 tablet_replica.cc:330] T a836ea5c1efe4a7f9f72300d5c616410 P 3a942d9b1c76420db11f4729624c6d3e: stopping tablet replica
I20260619 17:00:04.136229 563 tablet_service.cc:1514] Processing DeleteTablet for tablet a836ea5c1efe4a7f9f72300d5c616410 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:00:04 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:50760
I20260619 17:00:04.136332 670 raft_consensus.cc:2229] T a836ea5c1efe4a7f9f72300d5c616410 P 3a942d9b1c76420db11f4729624c6d3e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 17:00:04.136431 670 raft_consensus.cc:2258] T a836ea5c1efe4a7f9f72300d5c616410 P 3a942d9b1c76420db11f4729624c6d3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 17:00:04.136911 670 ts_tablet_manager.cc:1905] T a836ea5c1efe4a7f9f72300d5c616410 P 3a942d9b1c76420db11f4729624c6d3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 17:00:04.136915 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 32694
W20260619 17:00:04.145149 32599 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.231.129:43353 (error 108)
I20260619 17:00:04.145897 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 363
I20260619 17:00:04.147279 671 tablet_replica.cc:330] T a836ea5c1efe4a7f9f72300d5c616410 P ed97bccc64294756ad1468d0472888c7: stopping tablet replica
I20260619 17:00:04.147490 671 raft_consensus.cc:2229] T a836ea5c1efe4a7f9f72300d5c616410 P ed97bccc64294756ad1468d0472888c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 17:00:04.147600 671 raft_consensus.cc:2258] T a836ea5c1efe4a7f9f72300d5c616410 P ed97bccc64294756ad1468d0472888c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 17:00:04.149212 645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:04.149027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:43353 (local address 127.0.0.1:46798)
0619 17:00:04.149084 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:04.149142 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.129:43353: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 17:00:04.149372 32599 catalog_manager.cc:4584] TS 3a942d9b1c76420db11f4729624c6d3e (127.20.231.129:43353): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a836ea5c1efe4a7f9f72300d5c616410: Network error: Client connection negotiation failed: client connection to 127.20.231.129:43353: connect: Connection refused (error 111)
I20260619 17:00:04.153147 645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:04.152999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:35671 (local address 127.0.0.1:52332)
0619 17:00:04.153077 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:04.153106 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.130:35671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260619 17:00:04.153770 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 499
I20260619 17:00:04.161352 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 32581
I20260619 17:00:04.161417 645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:04.161282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:41227 (local address 127.0.0.1:50770)
0619 17:00:04.161341 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:04.161370 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.131:41227: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 17:00:04.167254 21406 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 17:00:04.502148 21406 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 17:00:04.502255 21406 mini_ranger.cc:66] Stopping Ranger...
I20260619 17:00:04.847586 21406 mini_ranger.cc:68] Stopped Ranger
2026-06-19 17:00:04.847 UTC [32167] LOG: received smart shutdown request
2026-06-19 17:00:04.851 UTC [32167] LOG: background worker "logical replication launcher" (PID 32176) exited with exit code 1
2026-06-19 17:00:04.851 UTC [32171] LOG: shutting down
2026-06-19 17:00:05.025 UTC [32167] LOG: database system is shut down
2026-06-19T17:00:05Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48564 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 17:00:05 dist-test-slave-vnbj krb5kdc[681](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 17:00:05 dist-test-slave-vnbj krb5kdc[681](info): set up 2 sockets
Jun 19 17:00:05 dist-test-slave-vnbj krb5kdc[681](info): commencing operation
krb5kdc: starting...
W20260619 17:00:06.203179 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 17:00:06 dist-test-slave-vnbj krb5kdc[681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T17:00:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T17:00:06Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.20.231.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.20.231.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.20.231.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/rangeradmin_127.20.231.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.20.231.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/rangeradmin_127.20.231.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.20.231.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.20.231.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.20.231.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/rangerlookup_127.20.231.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.20.231.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/rangerlookup_127.20.231.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.231.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.20.231.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.231.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/HTTP_127.20.231.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.231.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/HTTP_127.20.231.148@KRBTEST.COM.keytab.
I20260619 17:00:06.374022 21406 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskGfxJfS/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/postgres -l logfile start
2026-06-19 17:00:09.050 UTC [711] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 17:00:09.050 UTC [711] LOG: listening on IPv4 address "127.20.231.148", port 37777
2026-06-19 17:00:09.054 UTC [711] LOG: listening on Unix socket "/tmp/.s.PGSQL.37777"
2026-06-19 17:00:09.083 UTC [714] LOG: database system was shut down at 2026-06-19 17:00:07 UTC
2026-06-19 17:00:09.088 UTC [711] LOG: database system is ready to accept connections
I20260619 17:00:10.054677 21406 mini_postgres.cc:96] Postgres bound to 37777
2026-06-19 17:00:10.061 UTC [722] FATAL: database "slave" does not exist
127.20.231.148:37777 - accepting connections
I20260619 17:00:10.062107 21406 mini_ranger.cc:161] Starting Ranger...
I20260619 17:00:10.077230 21406 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 17:00:10.621694 21406 mini_ranger.cc:87] Created ranger Postgres database
I20260619 17:00:10.621781 21406 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ranger-admin
2026-06-19 17:00:10,905 [I] DB FLAVOR :POSTGRES
2026-06-19 17:00:10,906 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 17:00:10,906 [I] Checking connection..
2026-06-19 17:00:10,906 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:37777/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 17:00:11,146 [I] Checking connection passed.
2026-06-19 17:00:11,147 [I] --------- Verifying version history table ---------
2026-06-19 17:00:11,147 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:37777/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 17:00:11,400 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 17:00:11,401 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:37777/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 17:00:11,666 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 17:00:11,666 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 17:00:11,666 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:37777/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 17:00:11.911 UTC [786] WARNING: there is no transaction in progress
2026-06-19 17:00:11,918 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 17:00:11,918 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:37777/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 17:00:12,158 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 17:00:12,158 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 17:00:12,159 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:37777/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 17:00:12,414 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:37777/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 17:00:12,710 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:37777/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-vnbj.c.gcp-upstream.internal','N') ;"
2026-06-19 17:00:12,962 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:37777/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 17:00:13,213 [I] Table x_portal_user does not exist in database ranger
2026-06-19 17:00:13,214 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:37777/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 17:00:13,482 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 17:00:13,483 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:37777/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 17:00:13,756 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:37777/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 17:00:14,010 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 17:00:14,010 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:37777/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 17:00:15.062 UTC [894] WARNING: there is no transaction in progress
2026-06-19 17:00:15.080 UTC [894] WARNING: there is no transaction in progress
2026-06-19 17:00:15.099 UTC [894] WARNING: there is no transaction in progress
2026-06-19 17:00:15.115 UTC [894] WARNING: there is no transaction in progress
2026-06-19 17:00:15.132 UTC [894] WARNING: there is no transaction in progress
2026-06-19 17:00:15.154 UTC [894] WARNING: there is no transaction in progress
2026-06-19 17:00:15.216 UTC [894] WARNING: there is no transaction in progress
2026-06-19 17:00:15.223 UTC [894] WARNING: there is no transaction in progress
2026-06-19 17:00:15.238 UTC [894] WARNING: there is no transaction in progress
2026-06-19 17:00:15.249 UTC [894] WARNING: there is no transaction in progress
2026-06-19 17:00:15.260 UTC [894] WARNING: there is no transaction in progress
2026-06-19 17:00:15.275 UTC [894] WARNING: there is no transaction in progress
2026-06-19 17:00:15.285 UTC [894] WARNING: there is no transaction in progress
2026-06-19 17:00:15.291 UTC [894] WARNING: there is no transaction in progress
2026-06-19 17:00:15.299 UTC [894] WARNING: there is no transaction in progress
2026-06-19 17:00:15.306 UTC [894] WARNING: there is no transaction in progress
2026-06-19 17:00:16,018 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 17:00:16,019 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:37777/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 17:00:16,290 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 17:00:16,291 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:37777/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 17:00:16,582 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 17:00:16,582 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:37777/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 17:00:16,822 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 17:00:16,823 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:37777/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 17:00:17,071 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:37777/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 17:00:17,331 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:37777/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-vnbj.c.gcp-upstream.internal';"
2026-06-19 17:00:17,599 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 17:00:17,599 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:37777/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 17:00:17,851 [I] DB_PATCHES have already been applied
I20260619 17:00:17.857378 21406 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ranger-admin:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskGfxJfS/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 17:00:17.857445 21406 mini_ranger.cc:193] Using host: 127.20.231.148
I20260619 17:00:17.860752 21406 mini_ranger.cc:227] Ranger admin URL: http://127.20.231.148:55959
Jun 19, 2026 5:00:18 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 5:00:18 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 5:00:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 5:00:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 5:00:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 5:00:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 5:00:18 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 17:00:18.997378 21406 mini_ranger.cc:160] Time spent starting Ranger: real 8.935s user 0.000s sys 0.010s
Jun 19, 2026 5:00:39 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 5:00:39 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Jun 19, 2026 5:00:39 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 5:00:39 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 5:00:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 5:00:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 5:00:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 5:00:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 5:00:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 5:00:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 5:00:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 5:00:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 5:00:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 5:00:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 5:00:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 5:00:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 5:00:39 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 5:00:40 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 17:00:41.305790 21406 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.20.231.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.20.231.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.20.231.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/rangerkms_127.20.231.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.20.231.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/rangerkms_127.20.231.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.20.231.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.20.231.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.20.231.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/HTTP_127.20.231.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.20.231.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/HTTP_127.20.231.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 17:00:41 dist-test-slave-vnbj krb5kdc[681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888441, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 17:00:41.394256 21406 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 17:00:41.404611 21406 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 17:00:42.416443 21406 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 17:00:42.416534 21406 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ranger-kms
I20260619 17:00:42.416553 21406 mini_ranger_kms.cc:227] Using postgres at 127.20.231.148:37777
2026-06-19 17:00:42,619 [I] DB FLAVOR :POSTGRES
2026-06-19 17:00:42,620 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 17:00:42,620 [I] Checking connection
2026-06-19 17:00:42,620 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:37777/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 17:00:42,912 [I] connection success
2026-06-19 17:00:42,912 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 17:00:42,913 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 17:00:42,913 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:37777/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 17:00:43,199 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 17:00:43,199 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 17:00:43,199 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 17:00:43,200 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.20.231.148:37777/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 17:00:43,506 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 17:00:44.036758 21406 mini_ranger_kms.cc:310] Created kms service
I20260619 17:00:44.231300 21406 mini_ranger_kms.cc:326] Created kudu user
I20260619 17:00:44.277277 21406 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 17:00:44 dist-test-slave-vnbj krb5kdc[681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888441, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.20.231.148@KRBTEST.COM
I20260619 17:00:44.786242 21406 mini_ranger_kms.cc:384] Added ranger policy
I20260619 17:00:44.786381 21406 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ranger-kms:/tmp/dist-test-taskGfxJfS/build/release/bin/postgresql.jar:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskGfxJfS/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 17:00:44.786415 21406 mini_ranger_kms.cc:240] Using host: 127.20.231.148
I20260619 17:00:44.791479 21406 mini_ranger_kms.cc:276] Ranger KMS PID: 1075
I20260619 17:00:44.791567 21406 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.20.231.148:42573
Jun 19, 2026 5:00:45 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 5:00:45 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 5:00:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 5:00:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 5:00:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 5:00:46 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260619 17:00:47.134758 21406 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.741s user 0.006s sys 0.180s
I20260619 17:00:47.134886 21406 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 17:00:47.134932 21406 mini_ranger_kms.cc:401] 127.20.231.148:42573/kms/v1/keys
Jun 19 17:00:47 dist-test-slave-vnbj krb5kdc[681](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781888447, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.231.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 19 17:00:47 dist-test-slave-vnbj krb5kdc[681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888447, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.231.148@KRBTEST.COM for HTTP/127.20.231.148@KRBTEST.COM
Jun 19 17:00:48 dist-test-slave-vnbj krb5kdc[681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888447, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.20.231.148@KRBTEST.COM for HTTP/127.20.231.148@KRBTEST.COM
Jun 19 17:00:50 dist-test-slave-vnbj krb5kdc[681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.20.231.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
I20260619 17:00:50.518332 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:41497
--webserver_interface=127.20.231.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:36563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:41497
--ranger_config_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.231.148:42573/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:00:50.600970 1125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:00:50.601202 1125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:00:50.601226 1125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:00:50.602391 1125 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:00:50.602430 1125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:00:50.602445 1125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:00:50.602458 1125 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:00:50.602469 1125 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:00:50.604095 1125 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:36563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/master-0/wal
--ranger_kms_url=127.20.231.148:42573/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:41497
--ranger_config_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:41497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:00:50.604305 1125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:00:50.604489 1125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 17:00:50.610297 1125 server_base.cc:1034] running on GCE node
W20260619 17:00:50.610329 1130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:00:50.610355 1133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:00:50.610360 1131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:00:50.611065 1125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:00:50.611354 1125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:00:50.612534 1125 hybrid_clock.cc:648] HybridClock initialized: now 1781888450612522 us; error 51 us; skew 500 ppm
Jun 19 17:00:50 dist-test-slave-vnbj krb5kdc[681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888450, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:00:50.621400 1125 init.cc:377] Logged in from keytab as kudu/127.20.231.190@KRBTEST.COM (short username kudu)
I20260619 17:00:50.622004 1125 webserver.cc:460] Webserver started at http://127.20.231.190:33117/ using document root <none> and password file <none>
I20260619 17:00:50.622268 1125 fs_manager.cc:359] Metadata directory not provided
I20260619 17:00:50.622344 1125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:00:50.622531 1125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 17:00:50 dist-test-slave-vnbj krb5kdc[681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888450, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for HTTP/127.20.231.148@KRBTEST.COM
I20260619 17:00:50.688697 1125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "4f4212d9747442968b6f74a2fb4de929"
format_stamp: "Formatted at 2026-06-19 17:00:50 on dist-test-slave-vnbj"
server_key: "dba8aa4c13d911e464e38d59566126c7"
server_key_iv: "63109aaba4e6100e8d82eb324820ccdc"
server_key_version: "kuduclusterkey@0"
I20260619 17:00:50.689195 1125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "4f4212d9747442968b6f74a2fb4de929"
format_stamp: "Formatted at 2026-06-19 17:00:50 on dist-test-slave-vnbj"
server_key: "dba8aa4c13d911e464e38d59566126c7"
server_key_iv: "63109aaba4e6100e8d82eb324820ccdc"
server_key_version: "kuduclusterkey@0"
I20260619 17:00:50.691215 1125 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260619 17:00:50.704078 1140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:00:50.704380 1125 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:00:50.704492 1125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "4f4212d9747442968b6f74a2fb4de929"
format_stamp: "Formatted at 2026-06-19 17:00:50 on dist-test-slave-vnbj"
server_key: "dba8aa4c13d911e464e38d59566126c7"
server_key_iv: "63109aaba4e6100e8d82eb324820ccdc"
server_key_version: "kuduclusterkey@0"
I20260619 17:00:50.704594 1125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:00:50.727643 1125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:00:50.729996 1125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:00:50.730165 1125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:00:50.735361 1125 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:41497
I20260619 17:00:50.735462 1192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:41497 every 8 connection(s)
I20260619 17:00:50.735849 1125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 17:00:50.736580 1193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:00:50.738961 1193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929: Bootstrap starting.
I20260619 17:00:50.739559 1193 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929: Neither blocks nor log segments found. Creating new log.
I20260619 17:00:50.739861 1193 log.cc:824] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929: Log is configured to *not* fsync() on all Append() calls
I20260619 17:00:50.740571 1193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929: No bootstrap required, opened a new log
I20260619 17:00:50.742535 1193 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f4212d9747442968b6f74a2fb4de929" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41497 } }
I20260619 17:00:50.742715 1193 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:00:50.742765 1193 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f4212d9747442968b6f74a2fb4de929, State: Initialized, Role: FOLLOWER
I20260619 17:00:50.743000 1193 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f4212d9747442968b6f74a2fb4de929" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41497 } }
I20260619 17:00:50.743108 1193 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 17:00:50.743152 1193 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 17:00:50.743199 1193 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:00:50.743767 1193 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f4212d9747442968b6f74a2fb4de929" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41497 } }
I20260619 17:00:50.743899 1193 leader_election.cc:302] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f4212d9747442968b6f74a2fb4de929; no voters:
I20260619 17:00:50.744098 1193 leader_election.cc:288] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 17:00:50.744163 1196 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:00:50.744372 1196 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929 [term 1 LEADER]: Becoming Leader. State: Replica: 4f4212d9747442968b6f74a2fb4de929, State: Running, Role: LEADER
I20260619 17:00:50.744395 1193 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:00:50.744534 1196 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f4212d9747442968b6f74a2fb4de929" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41497 } }
I20260619 17:00:50.745162 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 1125
I20260619 17:00:50.745282 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 17:00:50.745503 21406 external_mini_cluster.cc:1428] Setting key f182806639f33bce4ec9a7737c4b0ced
I20260619 17:00:50.746141 1197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f4212d9747442968b6f74a2fb4de929. Latest consensus state: current_term: 1 leader_uuid: "4f4212d9747442968b6f74a2fb4de929" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f4212d9747442968b6f74a2fb4de929" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41497 } } }
I20260619 17:00:50.746192 1198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f4212d9747442968b6f74a2fb4de929" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f4212d9747442968b6f74a2fb4de929" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41497 } } }
I20260619 17:00:50.746156 1193 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 17:00:50.746269 1197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929 [sys.catalog]: This master's current role is: LEADER
I20260619 17:00:50.746281 1198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929 [sys.catalog]: This master's current role is: LEADER
I20260619 17:00:50.746496 1214 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 17:00:50.749444 1214 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 17:00:50.752592 1214 catalog_manager.cc:1347] Generated new cluster ID: 5571b52e2ffa4f9781ea30ddb6bc8f62
I20260619 17:00:50.752713 1214 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 17:00:50 dist-test-slave-vnbj krb5kdc[681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:00:50.760192 1213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:50.746075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60762 (local address 127.20.231.190:41497)
0619 17:00:50.746240 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:50.746249 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:50.746435 (+ 186us) server_negotiation.cc:407] Connection header received
0619 17:00:50.746694 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:50.746701 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:50.748335 (+ 1634us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:50.748654 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:00:50.749530 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:50.751172 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:50.752082 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:50.752705 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:50.752819 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:50.755697 (+ 2878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:00:50.755702 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:00:50.755702 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:00:50.755716 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:00:50.758154 (+ 2438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:50.758585 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:50.758588 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:50.758589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:50.758696 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:50.758893 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:50.758893 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:50.758894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:50.759210 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:00:50.759770 (+ 560us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:50.759843 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 17:00:50.759969 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":75,"threads_started":1}
I20260619 17:00:50.760730 1214 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 17:00:50.761415 1214 catalog_manager.cc:1504] Loading token signing keys...
I20260619 17:00:50.771888 1214 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929: Generated new TSK 0
I20260619 17:00:50.772158 1214 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.231.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab.
Entry for principal kudu/127.20.231.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab.
I20260619 17:00:51.490151 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.231.148:42573/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.129:0
--local_ip_for_outbound_sockets=127.20.231.129
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:41497
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:36563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:00:51.580080 1239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:00:51.580334 1239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:00:51.580375 1239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:00:51.581667 1239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:00:51.581748 1239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:00:51.581794 1239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.129
I20260619 17:00:51.583438 1239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:36563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.20.231.148:42573/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:41497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.129
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:00:51.583719 1239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:00:51.584026 1239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:00:51.589383 1245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:00:51.589592 1244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:00:51.589520 1247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:00:51.589869 1239 server_base.cc:1034] running on GCE node
I20260619 17:00:51.590143 1239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:00:51.590404 1239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:00:51.591548 1239 hybrid_clock.cc:648] HybridClock initialized: now 1781888451591532 us; error 36 us; skew 500 ppm
Jun 19 17:00:51 dist-test-slave-vnbj krb5kdc[681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888451, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:00:51.594213 1239 init.cc:377] Logged in from keytab as kudu/127.20.231.129@KRBTEST.COM (short username kudu)
I20260619 17:00:51.594705 1239 webserver.cc:460] Webserver started at http://127.20.231.129:37091/ using document root <none> and password file <none>
I20260619 17:00:51.595057 1239 fs_manager.cc:359] Metadata directory not provided
I20260619 17:00:51.595134 1239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:00:51.595283 1239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 17:00:51 dist-test-slave-vnbj krb5kdc[681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888451, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for HTTP/127.20.231.148@KRBTEST.COM
I20260619 17:00:51.608968 1239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-0/data/instance:
uuid: "52b4fd0e03f44784a1a308be185e8967"
format_stamp: "Formatted at 2026-06-19 17:00:51 on dist-test-slave-vnbj"
server_key: "05bc7f9bfa006d47e86bc53c99ff98ca"
server_key_iv: "8782d50605503f6ad1771c6d9b292d1f"
server_key_version: "kuduclusterkey@0"
I20260619 17:00:51.609598 1239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance:
uuid: "52b4fd0e03f44784a1a308be185e8967"
format_stamp: "Formatted at 2026-06-19 17:00:51 on dist-test-slave-vnbj"
server_key: "05bc7f9bfa006d47e86bc53c99ff98ca"
server_key_iv: "8782d50605503f6ad1771c6d9b292d1f"
server_key_version: "kuduclusterkey@0"
I20260619 17:00:51.611515 1239 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260619 17:00:51.627439 1254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:00:51.627630 1239 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 17:00:51.627727 1239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-0/wal
uuid: "52b4fd0e03f44784a1a308be185e8967"
format_stamp: "Formatted at 2026-06-19 17:00:51 on dist-test-slave-vnbj"
server_key: "05bc7f9bfa006d47e86bc53c99ff98ca"
server_key_iv: "8782d50605503f6ad1771c6d9b292d1f"
server_key_version: "kuduclusterkey@0"
I20260619 17:00:51.627830 1239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:00:51.649442 1239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:00:51.651996 1239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:00:51.652143 1239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:00:51.652356 1239 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:00:51.652774 1239 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:00:51.652904 1239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:00:51.652941 1239 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:00:51.652988 1239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:00:51.659873 1239 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.129:35425
I20260619 17:00:51.660389 1239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
I20260619 17:00:51.662154 1367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.129:35425 every 8 connection(s)
I20260619 17:00:51.667527 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 1239
I20260619 17:00:51.667636 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance
I20260619 17:00:51.667824 21406 external_mini_cluster.cc:1428] Setting key 2f9655b1d02a476dc241ef16b3d5b2e0
Jun 19 17:00:51 dist-test-slave-vnbj krb5kdc[681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888451, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:00:51.675333 1371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:51.661432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:46677 (local address 127.20.231.190:41497)
0619 17:00:51.661603 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:51.661606 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:51.663112 (+ 1506us) server_negotiation.cc:407] Connection header received
0619 17:00:51.663786 (+ 674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:51.663787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:51.663799 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:51.663923 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:00:51.664671 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:51.665214 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:51.666678 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:51.666947 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:51.667000 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:51.671285 (+ 4285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:00:51.671287 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:00:51.671287 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:00:51.671296 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:00:51.673349 (+ 2053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:51.674014 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:51.674016 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:51.674017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:51.674091 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:51.674418 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:51.674419 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:51.674420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:51.674588 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:00:51.674667 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:51.675085 (+ 418us) server_negotiation.cc:299] Negotiation successful
0619 17:00:51.675194 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":93,"threads_started":1}
I20260619 17:00:51.676259 1370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:51.661168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41497 (local address 127.20.231.129:46677)
0619 17:00:51.662958 (+ 1790us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:51.663008 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 17:00:51.663677 (+ 669us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:00:51.664029 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:00:51.664034 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:00:51.664293 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:00:51.664575 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:51.664581 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:51.665308 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:51.665309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:51.666593 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:51.666595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:51.667036 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:51.667037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:51.667173 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:51.668336 (+ 1163us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:00:51.668346 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:00:51.671170 (+ 2824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:00:51.673467 (+ 2297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:00:51.673477 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:00:51.673480 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:00:51.673932 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:00:51.674203 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:00:51.674204 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:00:51.674204 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:00:51.674335 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:00:51.674662 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:00:51.674666 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:00:51.674794 (+ 128us) client_negotiation.cc:769] Sending connection context
0619 17:00:51.675719 (+ 925us) client_negotiation.cc:240] Negotiation successful
0619 17:00:51.675947 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":108,"threads_started":1}
I20260619 17:00:51.677289 1368 heartbeater.cc:347] Connected to a master server at 127.20.231.190:41497
I20260619 17:00:51.677402 1368 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.20.231.130@KRBTEST.COM; defaulting to no policy
I20260619 17:00:51.677600 1368 heartbeater.cc:511] Master 127.20.231.190:41497 requested a full tablet report, sending...
I20260619 17:00:51.678362 1157 ts_manager.cc:194] Registered new tserver with Master: 52b4fd0e03f44784a1a308be185e8967 (127.20.231.129:35425)
I20260619 17:00:51.679351 1157 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.129@KRBTEST.COM'} at 127.20.231.129:46677
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab.
Entry for principal kudu/127.20.231.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab.
I20260619 17:00:51.698527 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.231.148:42573/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.130:0
--local_ip_for_outbound_sockets=127.20.231.130
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:41497
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:36563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:00:51.804647 1376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:00:51.805049 1376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:00:51.805146 1376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:00:51.807359 1376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:00:51.807488 1376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:00:51.807581 1376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.130
I20260619 17:00:51.810214 1376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:36563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.20.231.148:42573/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:41497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.130
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:00:51.810508 1376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:00:51.810894 1376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:00:51.817654 1386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:00:51.817667 1384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:00:51.817955 1383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:00:51.818984 1376 server_base.cc:1034] running on GCE node
I20260619 17:00:51.819248 1376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:00:51.819573 1376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:00:51.820777 1376 hybrid_clock.cc:648] HybridClock initialized: now 1781888451820756 us; error 42 us; skew 500 ppm
Jun 19 17:00:51 dist-test-slave-vnbj krb5kdc[681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888451, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:00:51.824497 1376 init.cc:377] Logged in from keytab as kudu/127.20.231.130@KRBTEST.COM (short username kudu)
I20260619 17:00:51.825140 1376 webserver.cc:460] Webserver started at http://127.20.231.130:39387/ using document root <none> and password file <none>
I20260619 17:00:51.825452 1376 fs_manager.cc:359] Metadata directory not provided
I20260619 17:00:51.825531 1376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:00:51.825695 1376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 17:00:51 dist-test-slave-vnbj krb5kdc[681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888451, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for HTTP/127.20.231.148@KRBTEST.COM
I20260619 17:00:51.841286 1376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-1/data/instance:
uuid: "42df110a75084262aa072a23bd7b768e"
format_stamp: "Formatted at 2026-06-19 17:00:51 on dist-test-slave-vnbj"
server_key: "90f3157f864f74e9d5de1c492c6f1b07"
server_key_iv: "928a088442f63f8dfbd8667555789bc6"
server_key_version: "kuduclusterkey@0"
I20260619 17:00:51.841939 1376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance:
uuid: "42df110a75084262aa072a23bd7b768e"
format_stamp: "Formatted at 2026-06-19 17:00:51 on dist-test-slave-vnbj"
server_key: "90f3157f864f74e9d5de1c492c6f1b07"
server_key_iv: "928a088442f63f8dfbd8667555789bc6"
server_key_version: "kuduclusterkey@0"
I20260619 17:00:51.847322 1376 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260619 17:00:51.872061 1393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:00:51.872489 1376 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 17:00:51.872718 1376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-1/wal
uuid: "42df110a75084262aa072a23bd7b768e"
format_stamp: "Formatted at 2026-06-19 17:00:51 on dist-test-slave-vnbj"
server_key: "90f3157f864f74e9d5de1c492c6f1b07"
server_key_iv: "928a088442f63f8dfbd8667555789bc6"
server_key_version: "kuduclusterkey@0"
I20260619 17:00:51.872898 1376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Jun 19 17:00:51 dist-test-slave-vnbj krb5kdc[681](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781888451, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:00:51.922628 1376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:00:51.925756 1376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:00:51.926048 1376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:00:51.926368 1376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:00:51.926823 1376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:00:51.926952 1376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:00:51.927023 1376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:00:51.927081 1376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:00:51.933645 1376 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.130:34947
I20260619 17:00:51.934155 1376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
I20260619 17:00:51.935379 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 1376
I20260619 17:00:51.935501 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance
I20260619 17:00:51.935721 21406 external_mini_cluster.cc:1428] Setting key bad93f55ac655ec3fff436630645312d
I20260619 17:00:51.937762 1506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.130:34947 every 8 connection(s)
Jun 19 17:00:51 dist-test-slave-vnbj krb5kdc[681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888451, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:00:51.950060 1371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:51.938621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:35239 (local address 127.20.231.190:41497)
0619 17:00:51.938671 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:51.938673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:51.938963 (+ 290us) server_negotiation.cc:407] Connection header received
0619 17:00:51.939704 (+ 741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:51.939705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:51.939722 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:51.939755 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:00:51.940644 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:51.941463 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:51.943050 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:51.943390 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:51.943435 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:51.946222 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:00:51.946224 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:00:51.946224 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:00:51.946234 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:00:51.948187 (+ 1953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:51.948758 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:51.948759 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:51.948760 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:51.948821 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:51.949220 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:51.949221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:51.949222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:51.949381 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:00:51.949449 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:51.949854 (+ 405us) server_negotiation.cc:299] Negotiation successful
0619 17:00:51.949954 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 17:00:51.950706 1510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:51.938631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41497 (local address 127.20.231.130:35239)
0619 17:00:51.938796 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:51.938841 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 17:00:51.939558 (+ 717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:00:51.939954 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:00:51.939959 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:00:51.940226 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:00:51.940526 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:51.940533 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:51.941587 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:51.941587 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:51.942948 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:51.942951 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:51.943539 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:51.943540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:51.943688 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:51.944367 (+ 679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:00:51.944378 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:00:51.946102 (+ 1724us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:00:51.948312 (+ 2210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:00:51.948329 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:00:51.948332 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:00:51.948660 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:00:51.948991 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:00:51.948992 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:00:51.948992 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:00:51.949143 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:00:51.949498 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:00:51.949505 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:00:51.949691 (+ 186us) client_negotiation.cc:769] Sending connection context
0619 17:00:51.950222 (+ 531us) client_negotiation.cc:240] Negotiation successful
0619 17:00:51.950417 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":95,"threads_started":1}
I20260619 17:00:51.951196 1507 heartbeater.cc:347] Connected to a master server at 127.20.231.190:41497
I20260619 17:00:51.951305 1507 heartbeater.cc:464] Registering TS with master...
I20260619 17:00:51.951536 1507 heartbeater.cc:511] Master 127.20.231.190:41497 requested a full tablet report, sending...
I20260619 17:00:51.951910 1157 ts_manager.cc:194] Registered new tserver with Master: 42df110a75084262aa072a23bd7b768e (127.20.231.130:34947)
I20260619 17:00:51.952442 1157 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.130@KRBTEST.COM'} at 127.20.231.130:35239
WARNING: no policy specified for kudu/127.20.231.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab.
Entry for principal kudu/127.20.231.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab.
I20260619 17:00:51.981942 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.231.148:42573/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.131:0
--local_ip_for_outbound_sockets=127.20.231.131
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:41497
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:36563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:00:52.106786 1512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:00:52.107208 1512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:00:52.107303 1512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:00:52.109462 1512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:00:52.109581 1512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:00:52.109683 1512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.131
I20260619 17:00:52.112154 1512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:36563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.20.231.148:42573/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:41497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1512
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.131
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:00:52.112680 1512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:00:52.113035 1512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:00:52.121328 1521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:00:52.121326 1518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:00:52.123481 1519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:00:52.132694 1512 server_base.cc:1034] running on GCE node
I20260619 17:00:52.133196 1512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:00:52.133819 1512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:00:52.137108 1512 hybrid_clock.cc:648] HybridClock initialized: now 1781888452137081 us; error 44 us; skew 500 ppm
Jun 19 17:00:52 dist-test-slave-vnbj krb5kdc[681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888452, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:00:52.140888 1512 init.cc:377] Logged in from keytab as kudu/127.20.231.131@KRBTEST.COM (short username kudu)
I20260619 17:00:52.141481 1512 webserver.cc:460] Webserver started at http://127.20.231.131:44177/ using document root <none> and password file <none>
I20260619 17:00:52.142275 1512 fs_manager.cc:359] Metadata directory not provided
I20260619 17:00:52.142396 1512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:00:52.142586 1512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 17:00:52 dist-test-slave-vnbj krb5kdc[681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888452, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for HTTP/127.20.231.148@KRBTEST.COM
I20260619 17:00:52.161983 1512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-2/data/instance:
uuid: "ef2f3094aed44ce1b1eb1db5cd7065f6"
format_stamp: "Formatted at 2026-06-19 17:00:52 on dist-test-slave-vnbj"
server_key: "c0b3e2cdf5dec8471fbdc4a02794b258"
server_key_iv: "bb8c653d3dffccc12a6d936b49846567"
server_key_version: "kuduclusterkey@0"
I20260619 17:00:52.162545 1512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance:
uuid: "ef2f3094aed44ce1b1eb1db5cd7065f6"
format_stamp: "Formatted at 2026-06-19 17:00:52 on dist-test-slave-vnbj"
server_key: "c0b3e2cdf5dec8471fbdc4a02794b258"
server_key_iv: "bb8c653d3dffccc12a6d936b49846567"
server_key_version: "kuduclusterkey@0"
I20260619 17:00:52.165800 1512 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.000s
I20260619 17:00:52.186414 1528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:00:52.186637 1512 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 17:00:52.186735 1512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-2/wal
uuid: "ef2f3094aed44ce1b1eb1db5cd7065f6"
format_stamp: "Formatted at 2026-06-19 17:00:52 on dist-test-slave-vnbj"
server_key: "c0b3e2cdf5dec8471fbdc4a02794b258"
server_key_iv: "bb8c653d3dffccc12a6d936b49846567"
server_key_version: "kuduclusterkey@0"
I20260619 17:00:52.186873 1512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:00:52.216032 1512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:00:52.218560 1512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:00:52.218716 1512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:00:52.219018 1512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:00:52.219426 1512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:00:52.219481 1512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:00:52.219514 1512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:00:52.219540 1512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:00:52.227672 1512 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.131:42009
I20260619 17:00:52.228233 1512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
I20260619 17:00:52.233201 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 1512
I20260619 17:00:52.233307 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance
I20260619 17:00:52.233507 21406 external_mini_cluster.cc:1428] Setting key ea99c8e7dff4e26d3597ee8a0dbe9872
I20260619 17:00:52.230993 1641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.131:42009 every 8 connection(s)
Jun 19 17:00:52 dist-test-slave-vnbj krb5kdc[681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888452, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:00:52.264477 1371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:52.229220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:49967 (local address 127.20.231.190:41497)
0619 17:00:52.229283 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:52.229285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:52.229633 (+ 348us) server_negotiation.cc:407] Connection header received
0619 17:00:52.230455 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:52.230457 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:52.230471 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:52.230538 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:00:52.231477 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:52.232316 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:52.234222 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:52.234563 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:52.234618 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:52.237666 (+ 3048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:00:52.237668 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:00:52.237669 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:00:52.237734 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:00:52.243780 (+ 6046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:52.263014 (+ 19234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:52.263016 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:52.263017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:52.263135 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:52.263575 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:52.263576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:52.263577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:52.263740 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:00:52.263801 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:52.264238 (+ 437us) server_negotiation.cc:299] Negotiation successful
0619 17:00:52.264376 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 17:00:52.265115 1644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:52.229192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41497 (local address 127.20.231.131:49967)
0619 17:00:52.229437 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:52.229499 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0619 17:00:52.230302 (+ 803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:00:52.230643 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:00:52.230648 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:00:52.230989 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:00:52.231349 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:52.231358 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:52.232442 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:52.232444 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:52.234106 (+ 1662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:52.234110 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:52.234748 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:52.234750 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:52.234967 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:52.235717 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:00:52.235729 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:00:52.237537 (+ 1808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:00:52.257338 (+ 19801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:00:52.257355 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:00:52.257359 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:00:52.257801 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:00:52.263253 (+ 5452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:00:52.263254 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:00:52.263254 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:00:52.263478 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:00:52.263838 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:00:52.263848 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:00:52.264051 (+ 203us) client_negotiation.cc:769] Sending connection context
0619 17:00:52.264653 (+ 602us) client_negotiation.cc:240] Negotiation successful
0619 17:00:52.264857 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":94,"threads_started":1}
I20260619 17:00:52.265615 1642 heartbeater.cc:347] Connected to a master server at 127.20.231.190:41497
I20260619 17:00:52.265725 1642 heartbeater.cc:464] Registering TS with master...
I20260619 17:00:52.265939 1642 heartbeater.cc:511] Master 127.20.231.190:41497 requested a full tablet report, sending...
I20260619 17:00:52.266314 1157 ts_manager.cc:194] Registered new tserver with Master: ef2f3094aed44ce1b1eb1db5cd7065f6 (127.20.231.131:42009)
I20260619 17:00:52.266803 1157 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.131@KRBTEST.COM'} at 127.20.231.131:49967
I20260619 17:00:52.266881 21406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 17:00:52.267743 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 1239
I20260619 17:00:52.272621 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 1376
I20260619 17:00:52.280043 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 1512
I20260619 17:00:52.285609 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 1125
I20260619 17:00:52.292129 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:41497
--webserver_interface=127.20.231.190
--webserver_port=33117
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:36563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:41497
--ranger_config_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.231.148:42573/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.231.148:42573/kms with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:00:52.376899 1645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:00:52.377128 1645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:00:52.377151 1645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:00:52.378458 1645 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:00:52.378509 1645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:00:52.378525 1645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:00:52.378537 1645 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:00:52.378549 1645 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:00:52.380329 1645 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:36563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.231.148:42573/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:41497
--ranger_config_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:41497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=33117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:00:52.380582 1645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:00:52.380822 1645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:00:52.386030 1653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:00:52.386302 1645 server_base.cc:1034] running on GCE node
W20260619 17:00:52.386034 1650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:00:52.386030 1651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:00:52.386761 1645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:00:52.387194 1645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:00:52.388381 1645 hybrid_clock.cc:648] HybridClock initialized: now 1781888452388359 us; error 34 us; skew 500 ppm
Jun 19 17:00:52 dist-test-slave-vnbj krb5kdc[681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888452, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:00:52.391263 1645 init.cc:377] Logged in from keytab as kudu/127.20.231.190@KRBTEST.COM (short username kudu)
I20260619 17:00:52.391829 1645 webserver.cc:460] Webserver started at http://127.20.231.190:33117/ using document root <none> and password file <none>
I20260619 17:00:52.392079 1645 fs_manager.cc:359] Metadata directory not provided
I20260619 17:00:52.392146 1645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 17:00:52 dist-test-slave-vnbj krb5kdc[681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888452, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for HTTP/127.20.231.148@KRBTEST.COM
I20260619 17:00:52.435961 1645 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 17:00:52.436872 1661 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:00:52.437100 1645 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 17:00:52.437206 1645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "4f4212d9747442968b6f74a2fb4de929"
format_stamp: "Formatted at 2026-06-19 17:00:50 on dist-test-slave-vnbj"
server_key: "dba8aa4c13d911e464e38d59566126c7"
server_key_iv: "63109aaba4e6100e8d82eb324820ccdc"
server_key_version: "kuduclusterkey@0"
I20260619 17:00:52.437681 1645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:00:52.470724 1645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:00:52.472997 1645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:00:52.473119 1645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:00:52.477739 1645 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:41497
I20260619 17:00:52.477782 1713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:41497 every 8 connection(s)
I20260619 17:00:52.478185 1645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 17:00:52.479126 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 1645
I20260619 17:00:52.479431 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.231.148:42573/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.129:35425
--local_ip_for_outbound_sockets=127.20.231.129
--tserver_master_addrs=127.20.231.190:41497
--webserver_port=37091
--webserver_interface=127.20.231.129
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:36563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.231.148:42573/kms with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 17:00:52.479734 1714 sys_catalog.cc:263] Verifying existing consensus state
I20260619 17:00:52.480379 1714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929: Bootstrap starting.
I20260619 17:00:52.488821 1714 log.cc:824] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929: Log is configured to *not* fsync() on all Append() calls
I20260619 17:00:52.490826 1714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 17:00:52.491225 1714 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929: Bootstrap complete.
I20260619 17:00:52.493646 1714 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f4212d9747442968b6f74a2fb4de929" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41497 } }
I20260619 17:00:52.493906 1714 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f4212d9747442968b6f74a2fb4de929, State: Initialized, Role: FOLLOWER
I20260619 17:00:52.494087 1714 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f4212d9747442968b6f74a2fb4de929" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41497 } }
I20260619 17:00:52.494168 1714 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 17:00:52.494222 1714 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 17:00:52.494273 1714 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929 [term 1 FOLLOWER]: Advancing to term 2
I20260619 17:00:52.495433 1714 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f4212d9747442968b6f74a2fb4de929" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41497 } }
I20260619 17:00:52.495553 1714 leader_election.cc:302] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f4212d9747442968b6f74a2fb4de929; no voters:
I20260619 17:00:52.495806 1714 leader_election.cc:288] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 17:00:52.495848 1718 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 17:00:52.495961 1718 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929 [term 2 LEADER]: Becoming Leader. State: Replica: 4f4212d9747442968b6f74a2fb4de929, State: Running, Role: LEADER
I20260619 17:00:52.496076 1714 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:00:52.496076 1718 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f4212d9747442968b6f74a2fb4de929" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41497 } }
I20260619 17:00:52.496501 1719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4f4212d9747442968b6f74a2fb4de929" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f4212d9747442968b6f74a2fb4de929" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41497 } } }
I20260619 17:00:52.496598 1719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929 [sys.catalog]: This master's current role is: LEADER
I20260619 17:00:52.496498 1720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f4212d9747442968b6f74a2fb4de929. Latest consensus state: current_term: 2 leader_uuid: "4f4212d9747442968b6f74a2fb4de929" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f4212d9747442968b6f74a2fb4de929" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41497 } } }
I20260619 17:00:52.496891 1720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929 [sys.catalog]: This master's current role is: LEADER
I20260619 17:00:52.497030 1726 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 17:00:52.497525 1726 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 17:00:52.497645 1726 catalog_manager.cc:1259] Loaded cluster ID: 5571b52e2ffa4f9781ea30ddb6bc8f62
I20260619 17:00:52.497675 1726 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 17:00:52.497999 1714 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 17:00:52.501240 1726 catalog_manager.cc:1504] Loading token signing keys...
I20260619 17:00:52.501852 1726 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929: Loaded TSK: 0
I20260619 17:00:52.502148 1726 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 17:00:52.589658 1716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:00:52.589905 1716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:00:52.589946 1716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:00:52.591295 1716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:00:52.591346 1716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:00:52.591387 1716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.129
I20260619 17:00:52.593217 1716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:36563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.231.148:42573/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.129:35425
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.231.129
--webserver_port=37091
--tserver_master_addrs=127.20.231.190:41497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.129
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:00:52.593503 1716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:00:52.593753 1716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:00:52.599221 1752 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:00:52.599291 1751 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:00:52.599609 1754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:00:52.599858 1716 server_base.cc:1034] running on GCE node
I20260619 17:00:52.600066 1716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:00:52.600348 1716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:00:52.601511 1716 hybrid_clock.cc:648] HybridClock initialized: now 1781888452601500 us; error 31 us; skew 500 ppm
Jun 19 17:00:52 dist-test-slave-vnbj krb5kdc[681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888452, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:00:52.605180 1716 init.cc:377] Logged in from keytab as kudu/127.20.231.129@KRBTEST.COM (short username kudu)
I20260619 17:00:52.605778 1716 webserver.cc:460] Webserver started at http://127.20.231.129:37091/ using document root <none> and password file <none>
I20260619 17:00:52.606118 1716 fs_manager.cc:359] Metadata directory not provided
I20260619 17:00:52.606266 1716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 17:00:52 dist-test-slave-vnbj krb5kdc[681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888452, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for HTTP/127.20.231.148@KRBTEST.COM
I20260619 17:00:52.631832 1716 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.001s sys 0.000s
I20260619 17:00:52.632992 1762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:00:52.633227 1716 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:00:52.633322 1716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-0/wal
uuid: "52b4fd0e03f44784a1a308be185e8967"
format_stamp: "Formatted at 2026-06-19 17:00:51 on dist-test-slave-vnbj"
server_key: "05bc7f9bfa006d47e86bc53c99ff98ca"
server_key_iv: "8782d50605503f6ad1771c6d9b292d1f"
server_key_version: "kuduclusterkey@0"
I20260619 17:00:52.633796 1716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:00:52.658640 1716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:00:52.661127 1716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:00:52.661377 1716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:00:52.661612 1716 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:00:52.662009 1716 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:00:52.662104 1716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:00:52.662209 1716 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:00:52.662279 1716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:00:52.669867 1716 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.129:35425
I20260619 17:00:52.670104 1875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.129:35425 every 8 connection(s)
I20260619 17:00:52.670475 1716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
I20260619 17:00:52.673741 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 1716
I20260619 17:00:52.674062 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.231.148:42573/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.130:34947
--local_ip_for_outbound_sockets=127.20.231.130
--tserver_master_addrs=127.20.231.190:41497
--webserver_port=39387
--webserver_interface=127.20.231.130
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:36563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.231.148:42573/kms with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 17:00:52 dist-test-slave-vnbj krb5kdc[681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888452, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:00:52.682741 1879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:52.671487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:33019 (local address 127.20.231.190:41497)
0619 17:00:52.671662 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:52.671677 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:52.671707 (+ 30us) server_negotiation.cc:407] Connection header received
0619 17:00:52.672437 (+ 730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:52.672442 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:52.672519 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:52.672662 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:00:52.673334 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:52.674427 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:52.675836 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:52.676494 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:52.676604 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:52.678972 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:00:52.678975 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:00:52.678975 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:00:52.678985 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:00:52.680679 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:52.681377 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:52.681380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:52.681380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:52.681469 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:52.681748 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:52.681749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:52.681750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:52.682002 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:00:52.682107 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:52.682405 (+ 298us) server_negotiation.cc:299] Negotiation successful
0619 17:00:52.682525 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":80,"threads_started":1}
I20260619 17:00:52.682753 1878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:52.671324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41497 (local address 127.20.231.129:33019)
0619 17:00:52.671528 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:52.671580 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 17:00:52.672202 (+ 622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:00:52.672722 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:00:52.672726 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:00:52.672963 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:00:52.673238 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:52.673245 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:52.674536 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:52.674537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:52.675730 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:52.675734 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:52.676708 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:52.676709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:52.676798 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:52.677340 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:00:52.677349 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:00:52.678809 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:00:52.680835 (+ 2026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:00:52.680842 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:00:52.680844 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:00:52.681140 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:00:52.681554 (+ 414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:00:52.681555 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:00:52.681555 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:00:52.681671 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:00:52.682084 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:00:52.682090 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:00:52.682234 (+ 144us) client_negotiation.cc:769] Sending connection context
0619 17:00:52.682352 (+ 118us) client_negotiation.cc:240] Negotiation successful
0619 17:00:52.682525 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":107,"threads_started":1}
I20260619 17:00:52.684010 1876 heartbeater.cc:347] Connected to a master server at 127.20.231.190:41497
I20260619 17:00:52.684135 1876 heartbeater.cc:464] Registering TS with master...
I20260619 17:00:52.684330 1876 heartbeater.cc:511] Master 127.20.231.190:41497 requested a full tablet report, sending...
W20260619 17:00:52.685180 1876 heartbeater.cc:644] Failed to heartbeat to 127.20.231.190:41497 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 17:00:52.685272 1876 heartbeater.cc:464] Registering TS with master...
I20260619 17:00:52.685339 1876 heartbeater.cc:511] Master 127.20.231.190:41497 requested a full tablet report, sending...
I20260619 17:00:52.685627 1876 heartbeater.cc:464] Registering TS with master...
I20260619 17:00:52.685714 1876 heartbeater.cc:511] Master 127.20.231.190:41497 requested a full tablet report, sending...
W20260619 17:00:52.685971 1876 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 17:00:52.790683 1880 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:00:52.791069 1880 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:00:52.791147 1880 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:00:52.792480 1880 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:00:52.792567 1880 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:00:52.792616 1880 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.130
I20260619 17:00:52.794807 1880 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:36563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.231.148:42573/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.130:34947
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.231.130
--webserver_port=39387
--tserver_master_addrs=127.20.231.190:41497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1880
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.130
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:00:52.795089 1880 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:00:52.795367 1880 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:00:52.804049 1888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:00:52.804899 1880 server_base.cc:1034] running on GCE node
W20260619 17:00:52.807569 1885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:00:52.808447 1886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:00:52.808837 1880 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:00:52.809175 1880 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:00:52.810406 1880 hybrid_clock.cc:648] HybridClock initialized: now 1781888452810372 us; error 59 us; skew 500 ppm
Jun 19 17:00:52 dist-test-slave-vnbj krb5kdc[681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888452, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:00:52.813514 1880 init.cc:377] Logged in from keytab as kudu/127.20.231.130@KRBTEST.COM (short username kudu)
I20260619 17:00:52.814122 1880 webserver.cc:460] Webserver started at http://127.20.231.130:39387/ using document root <none> and password file <none>
I20260619 17:00:52.814357 1880 fs_manager.cc:359] Metadata directory not provided
I20260619 17:00:52.814453 1880 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 17:00:52 dist-test-slave-vnbj krb5kdc[681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888452, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for HTTP/127.20.231.148@KRBTEST.COM
I20260619 17:00:52.830696 1880 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:00:52.831854 1896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:00:52.832044 1880 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 17:00:52.832140 1880 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-1/wal
uuid: "42df110a75084262aa072a23bd7b768e"
format_stamp: "Formatted at 2026-06-19 17:00:51 on dist-test-slave-vnbj"
server_key: "90f3157f864f74e9d5de1c492c6f1b07"
server_key_iv: "928a088442f63f8dfbd8667555789bc6"
server_key_version: "kuduclusterkey@0"
I20260619 17:00:52.832649 1880 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:00:52.846551 1880 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:00:52.849090 1880 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:00:52.849285 1880 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:00:52.849615 1880 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:00:52.850032 1880 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:00:52.850095 1880 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:00:52.850138 1880 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:00:52.850165 1880 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:00:52.858417 1880 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.130:34947
I20260619 17:00:52.859027 1880 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
I20260619 17:00:52.859006 2009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.130:34947 every 8 connection(s)
I20260619 17:00:52.861503 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 1880
I20260619 17:00:52.861796 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.20.231.148:42573/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.131:42009
--local_ip_for_outbound_sockets=127.20.231.131
--tserver_master_addrs=127.20.231.190:41497
--webserver_port=44177
--webserver_interface=127.20.231.131
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:36563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.20.231.148:42573/kms with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 17:00:52 dist-test-slave-vnbj krb5kdc[681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888452, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:00:52.872181 2012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:52.859950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41497 (local address 127.20.231.130:33501)
0619 17:00:52.860280 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:52.860335 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0619 17:00:52.861084 (+ 749us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:00:52.861479 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:00:52.861484 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:00:52.861810 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:00:52.862387 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:52.862395 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:52.863562 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:52.863564 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:52.864613 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:52.864616 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:52.865105 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:52.865107 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:52.865255 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:52.865923 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:00:52.865932 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:00:52.867757 (+ 1825us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:00:52.870151 (+ 2394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:00:52.870161 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:00:52.870164 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:00:52.870530 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:00:52.870820 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:00:52.870821 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:00:52.870821 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:00:52.871025 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:00:52.871411 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:00:52.871418 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:00:52.871602 (+ 184us) client_negotiation.cc:769] Sending connection context
0619 17:00:52.871750 (+ 148us) client_negotiation.cc:240] Negotiation successful
0619 17:00:52.871936 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":148,"threads_started":1}
I20260619 17:00:52.872535 1879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:52.859956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:33501 (local address 127.20.231.190:41497)
0619 17:00:52.860007 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:52.860009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:52.860433 (+ 424us) server_negotiation.cc:407] Connection header received
0619 17:00:52.861239 (+ 806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:52.861241 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:52.861256 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:52.861332 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:00:52.862520 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:52.863426 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:52.864728 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:52.864965 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:52.865074 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:52.867871 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:00:52.867873 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:00:52.867874 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:00:52.867883 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:00:52.870026 (+ 2143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:52.870633 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:52.870634 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:52.870635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:52.870722 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:52.871118 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:52.871119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:52.871120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:52.871311 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:00:52.871403 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:52.872348 (+ 945us) server_negotiation.cc:299] Negotiation successful
0619 17:00:52.872442 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 17:00:52.872905 2010 heartbeater.cc:347] Connected to a master server at 127.20.231.190:41497
I20260619 17:00:52.873029 2010 heartbeater.cc:464] Registering TS with master...
I20260619 17:00:52.873252 2010 heartbeater.cc:511] Master 127.20.231.190:41497 requested a full tablet report, sending...
W20260619 17:00:52.874119 2010 heartbeater.cc:644] Failed to heartbeat to 127.20.231.190:41497 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 17:00:52.874218 2010 heartbeater.cc:464] Registering TS with master...
I20260619 17:00:52.874310 2010 heartbeater.cc:511] Master 127.20.231.190:41497 requested a full tablet report, sending...
I20260619 17:00:52.874778 2010 heartbeater.cc:464] Registering TS with master...
I20260619 17:00:52.874919 2010 heartbeater.cc:511] Master 127.20.231.190:41497 requested a full tablet report, sending...
W20260619 17:00:52.875348 2010 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 17:00:52.982805 2013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:00:52.983134 2013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:00:52.983175 2013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:00:52.984818 2013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:00:52.984872 2013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:00:52.984938 2013 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.131
I20260619 17:00:52.986609 2013 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:36563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.20.231.148:42573/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.131:42009
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.231.131
--webserver_port=44177
--tserver_master_addrs=127.20.231.190:41497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.131
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:00:52.986907 2013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:00:52.987174 2013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:00:52.992786 2018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:00:52.993075 2021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:00:52.994066 2013 server_base.cc:1034] running on GCE node
W20260619 17:00:52.995039 2019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:00:52.995337 2013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:00:52.995636 2013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:00:53.011111 2013 hybrid_clock.cc:648] HybridClock initialized: now 1781888453011067 us; error 60 us; skew 500 ppm
Jun 19 17:00:53 dist-test-slave-vnbj krb5kdc[681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888453, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:00:53.015641 2013 init.cc:377] Logged in from keytab as kudu/127.20.231.131@KRBTEST.COM (short username kudu)
I20260619 17:00:53.016311 2013 webserver.cc:460] Webserver started at http://127.20.231.131:44177/ using document root <none> and password file <none>
I20260619 17:00:53.016657 2013 fs_manager.cc:359] Metadata directory not provided
I20260619 17:00:53.016810 2013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 17:00:53 dist-test-slave-vnbj krb5kdc[681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888453, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for HTTP/127.20.231.148@KRBTEST.COM
I20260619 17:00:53.031960 2013 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:00:53.032786 2030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:00:53.033020 2013 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:00:53.033121 2013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-2/wal
uuid: "ef2f3094aed44ce1b1eb1db5cd7065f6"
format_stamp: "Formatted at 2026-06-19 17:00:52 on dist-test-slave-vnbj"
server_key: "c0b3e2cdf5dec8471fbdc4a02794b258"
server_key_iv: "bb8c653d3dffccc12a6d936b49846567"
server_key_version: "kuduclusterkey@0"
I20260619 17:00:53.033521 2013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:00:53.054803 2013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:00:53.058305 2013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:00:53.058507 2013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:00:53.058815 2013 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:00:53.059373 2013 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:00:53.059448 2013 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:00:53.059484 2013 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:00:53.059504 2013 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:00:53.067687 2013 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.131:42009
I20260619 17:00:53.067750 2143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.131:42009 every 8 connection(s)
I20260619 17:00:53.068132 2013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
I20260619 17:00:53.073645 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 2013
Jun 19 17:00:53 dist-test-slave-vnbj krb5kdc[681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888453, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:00:53.078678 1879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:53.068983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:44137 (local address 127.20.231.190:41497)
0619 17:00:53.069044 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:53.069046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:53.069346 (+ 300us) server_negotiation.cc:407] Connection header received
0619 17:00:53.069944 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:53.069945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:53.069958 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:53.070010 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:00:53.070631 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:53.071295 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:53.072425 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:53.072643 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:53.072754 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:53.075297 (+ 2543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:00:53.075299 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:00:53.075299 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:00:53.075307 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:00:53.077026 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:53.077595 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:53.077596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:53.077596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:53.077659 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:53.077941 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:53.077943 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:53.077943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:53.078069 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:00:53.078126 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:53.078504 (+ 378us) server_negotiation.cc:299] Negotiation successful
0619 17:00:53.078586 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 17:00:53.079098 2146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:53.068985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41497 (local address 127.20.231.131:44137)
0619 17:00:53.069190 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:53.069235 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 17:00:53.069823 (+ 588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:00:53.070085 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:00:53.070089 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:00:53.070304 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:00:53.070513 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:53.070518 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:53.071401 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:53.071402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:53.072300 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:53.072303 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:53.072774 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:53.072775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:53.072907 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:53.073554 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:00:53.073566 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:00:53.075198 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:00:53.077125 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:00:53.077133 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:00:53.077135 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:00:53.077518 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:00:53.077728 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:00:53.077729 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:00:53.077730 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:00:53.077855 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:00:53.078151 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:00:53.078157 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:00:53.078325 (+ 168us) client_negotiation.cc:769] Sending connection context
0619 17:00:53.078473 (+ 148us) client_negotiation.cc:240] Negotiation successful
0619 17:00:53.078634 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":132,"threads_started":1}
I20260619 17:00:53.079558 2144 heartbeater.cc:347] Connected to a master server at 127.20.231.190:41497
I20260619 17:00:53.079687 2144 heartbeater.cc:464] Registering TS with master...
I20260619 17:00:53.079881 2144 heartbeater.cc:511] Master 127.20.231.190:41497 requested a full tablet report, sending...
W20260619 17:00:53.080577 2144 heartbeater.cc:644] Failed to heartbeat to 127.20.231.190:41497 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 17:00:53.080663 2144 heartbeater.cc:464] Registering TS with master...
I20260619 17:00:53.080736 2144 heartbeater.cc:511] Master 127.20.231.190:41497 requested a full tablet report, sending...
I20260619 17:00:53.081019 2144 heartbeater.cc:464] Registering TS with master...
I20260619 17:00:53.081090 2144 heartbeater.cc:511] Master 127.20.231.190:41497 requested a full tablet report, sending...
W20260619 17:00:53.081399 2144 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 17:00:53.088169 2148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:53.074128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60768 (local address 127.20.231.190:41497)
0619 17:00:53.074467 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:53.074468 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:53.074474 (+ 6us) server_negotiation.cc:407] Connection header received
0619 17:00:53.074490 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:53.074491 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:53.074501 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:53.074550 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:00:53.075341 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:53.076029 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:53.077952 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:53.078250 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:53.078323 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:53.084814 (+ 6491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:00:53.084816 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:00:53.084816 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:00:53.084826 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:00:53.086731 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:53.087224 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:53.087225 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:53.087226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:53.087293 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:53.087563 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:53.087564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:53.087564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:53.087711 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:00:53.087890 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:53.087972 (+ 82us) server_negotiation.cc:299] Negotiation successful
0619 17:00:53.088061 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":77,"threads_started":1}
Jun 19 17:00:53 dist-test-slave-vnbj krb5kdc[681](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781888453, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:00:53.686163 1876 heartbeater.cc:464] Registering TS with master...
I20260619 17:00:53.686344 1876 heartbeater.cc:511] Master 127.20.231.190:41497 requested a full tablet report, sending...
I20260619 17:00:53.686733 1678 ts_manager.cc:194] Registered new tserver with Master: 52b4fd0e03f44784a1a308be185e8967 (127.20.231.129:35425)
I20260619 17:00:53.687463 1678 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.129@KRBTEST.COM'} at 127.20.231.129:33019
I20260619 17:00:53.875532 2010 heartbeater.cc:464] Registering TS with master...
I20260619 17:00:53.875702 2010 heartbeater.cc:511] Master 127.20.231.190:41497 requested a full tablet report, sending...
I20260619 17:00:53.876013 1678 ts_manager.cc:194] Registered new tserver with Master: 42df110a75084262aa072a23bd7b768e (127.20.231.130:34947)
I20260619 17:00:53.876438 1678 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.130@KRBTEST.COM'} at 127.20.231.130:33501
Jun 19 17:00:54 dist-test-slave-vnbj krb5kdc[681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888453, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for HTTP/127.20.231.148@KRBTEST.COM
I20260619 17:00:54.081614 2144 heartbeater.cc:464] Registering TS with master...
I20260619 17:00:54.081809 2144 heartbeater.cc:511] Master 127.20.231.190:41497 requested a full tablet report, sending...
I20260619 17:00:54.082129 1678 ts_manager.cc:194] Registered new tserver with Master: ef2f3094aed44ce1b1eb1db5cd7065f6 (127.20.231.131:42009)
I20260619 17:00:54.082643 1678 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.131@KRBTEST.COM'} at 127.20.231.131:44137
I20260619 17:00:54.084784 21406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 17:00:54.097729 2166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:54.087226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60784 (local address 127.20.231.190:41497)
0619 17:00:54.088462 (+ 1236us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:54.088464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:54.088472 (+ 8us) server_negotiation.cc:407] Connection header received
0619 17:00:54.088496 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:54.088497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:54.088508 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:54.088577 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:00:54.089385 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:54.090339 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:54.091603 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:54.091953 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:54.092016 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:54.093793 (+ 1777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:00:54.093794 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:00:54.093795 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:00:54.093804 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:00:54.095922 (+ 2118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:54.096442 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:54.096443 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:54.096444 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:54.096516 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:54.096718 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:54.096718 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:54.096719 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:54.096874 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:00:54.096977 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:54.097513 (+ 536us) server_negotiation.cc:299] Negotiation successful
0619 17:00:54.097609 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1215,"thread_start_us":69,"threads_started":1}
I20260619 17:00:54.100613 1678 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60784:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 17:00:54.103515 1678 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 17:00:54.119589 2173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:54.112590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:34947 (local address 127.0.0.1:46236)
0619 17:00:54.113539 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:54.113548 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 17:00:54.113592 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:00:54.114101 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:00:54.114102 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:00:54.114104 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:00:54.114149 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:54.114151 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:54.116728 (+ 2577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:54.116729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:54.118194 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:54.118196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:54.119233 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:54.119234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:54.119328 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:54.119343 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 17:00:54.119426 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 17:00:54.119489 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":934,"thread_start_us":66,"threads_started":1}
I20260619 17:00:54.119597 2171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:54.112343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:42009 (local address 127.0.0.1:52102)
0619 17:00:54.113126 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:54.113135 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 17:00:54.113184 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:00:54.114926 (+ 1742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:00:54.114927 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:00:54.114930 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:00:54.114978 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:54.114979 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:54.117373 (+ 2394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:54.117374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:54.118422 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:54.118424 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:54.119396 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:54.119397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:54.119464 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:54.119467 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 17:00:54.119509 (+ 42us) client_negotiation.cc:240] Negotiation successful
0619 17:00:54.119541 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":766,"mutex_wait_us":1,"thread_start_us":105,"threads_started":1}
I20260619 17:00:54.119966 2176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:54.113032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46236 (local address 127.20.231.130:34947)
0619 17:00:54.113485 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:54.113490 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:54.113643 (+ 153us) server_negotiation.cc:407] Connection header received
0619 17:00:54.113678 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:54.113680 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:54.113696 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:54.113761 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:00:54.114223 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:54.115562 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:54.118278 (+ 2716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:54.119064 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:54.119140 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:54.119196 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:54.119834 (+ 638us) server_negotiation.cc:299] Negotiation successful
0619 17:00:54.119887 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":57,"threads_started":1}
I20260619 17:00:54.120622 2172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:54.112343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:35425 (local address 127.0.0.1:37452)
0619 17:00:54.112839 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:54.112898 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0619 17:00:54.112975 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:00:54.113542 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:00:54.113546 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:00:54.113549 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:00:54.113626 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:54.113628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:54.115041 (+ 1413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:54.115042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:54.116342 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:54.116345 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:54.119621 (+ 3276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:54.119621 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:54.120435 (+ 814us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:54.120439 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 17:00:54.120487 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 17:00:54.120536 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":102,"threads_started":1}
I20260619 17:00:54.120791 2174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:54.112623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37452 (local address 127.20.231.129:35425)
0619 17:00:54.112770 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:54.112775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:54.113071 (+ 296us) server_negotiation.cc:407] Connection header received
0619 17:00:54.113346 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:54.113349 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:54.113366 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:54.113436 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:00:54.113704 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:54.114793 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:54.116437 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:54.117207 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:54.117284 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:54.117339 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:54.120646 (+ 3307us) server_negotiation.cc:299] Negotiation successful
0619 17:00:54.120717 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":53,"threads_started":1}
I20260619 17:00:54.120807 1944 tablet_service.cc:1467] Processing CreateTablet for tablet 312ba1fe180943208a291beb61757cdc (DEFAULT_TABLE table=test-table [id=3ea7361a9772465cbc049003e0853f4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 17:00:54.121054 2175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:54.112861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52102 (local address 127.20.231.131:42009)
0619 17:00:54.113887 (+ 1026us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:54.113891 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:54.113900 (+ 9us) server_negotiation.cc:407] Connection header received
0619 17:00:54.113930 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:54.113932 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:54.113947 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:54.114020 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:00:54.115658 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:54.116644 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:54.118510 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:54.119245 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:54.119309 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:54.119358 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:54.120928 (+ 1570us) server_negotiation.cc:299] Negotiation successful
0619 17:00:54.120980 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":994,"thread_start_us":74,"threads_started":1}
I20260619 17:00:54.121274 1944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 312ba1fe180943208a291beb61757cdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:00:54.121487 1792 tablet_service.cc:1467] Processing CreateTablet for tablet 312ba1fe180943208a291beb61757cdc (DEFAULT_TABLE table=test-table [id=3ea7361a9772465cbc049003e0853f4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 17:00:54.121582 2078 tablet_service.cc:1467] Processing CreateTablet for tablet 312ba1fe180943208a291beb61757cdc (DEFAULT_TABLE table=test-table [id=3ea7361a9772465cbc049003e0853f4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 17:00:54.121826 1792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 312ba1fe180943208a291beb61757cdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:00:54.121840 2078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 312ba1fe180943208a291beb61757cdc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:00:54.125866 2177 tablet_bootstrap.cc:492] T 312ba1fe180943208a291beb61757cdc P ef2f3094aed44ce1b1eb1db5cd7065f6: Bootstrap starting.
I20260619 17:00:54.126448 2177 tablet_bootstrap.cc:654] T 312ba1fe180943208a291beb61757cdc P ef2f3094aed44ce1b1eb1db5cd7065f6: Neither blocks nor log segments found. Creating new log.
I20260619 17:00:54.126762 2177 log.cc:824] T 312ba1fe180943208a291beb61757cdc P ef2f3094aed44ce1b1eb1db5cd7065f6: Log is configured to *not* fsync() on all Append() calls
I20260619 17:00:54.127125 2178 tablet_bootstrap.cc:492] T 312ba1fe180943208a291beb61757cdc P 52b4fd0e03f44784a1a308be185e8967: Bootstrap starting.
I20260619 17:00:54.127722 2177 tablet_bootstrap.cc:492] T 312ba1fe180943208a291beb61757cdc P ef2f3094aed44ce1b1eb1db5cd7065f6: No bootstrap required, opened a new log
I20260619 17:00:54.127741 2178 tablet_bootstrap.cc:654] T 312ba1fe180943208a291beb61757cdc P 52b4fd0e03f44784a1a308be185e8967: Neither blocks nor log segments found. Creating new log.
I20260619 17:00:54.127892 2177 ts_tablet_manager.cc:1397] T 312ba1fe180943208a291beb61757cdc P ef2f3094aed44ce1b1eb1db5cd7065f6: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 17:00:54.128038 2178 log.cc:824] T 312ba1fe180943208a291beb61757cdc P 52b4fd0e03f44784a1a308be185e8967: Log is configured to *not* fsync() on all Append() calls
I20260619 17:00:54.128389 2179 tablet_bootstrap.cc:492] T 312ba1fe180943208a291beb61757cdc P 42df110a75084262aa072a23bd7b768e: Bootstrap starting.
I20260619 17:00:54.129076 2179 tablet_bootstrap.cc:654] T 312ba1fe180943208a291beb61757cdc P 42df110a75084262aa072a23bd7b768e: Neither blocks nor log segments found. Creating new log.
I20260619 17:00:54.129388 2179 log.cc:824] T 312ba1fe180943208a291beb61757cdc P 42df110a75084262aa072a23bd7b768e: Log is configured to *not* fsync() on all Append() calls
I20260619 17:00:54.130656 2178 tablet_bootstrap.cc:492] T 312ba1fe180943208a291beb61757cdc P 52b4fd0e03f44784a1a308be185e8967: No bootstrap required, opened a new log
I20260619 17:00:54.130734 2178 ts_tablet_manager.cc:1397] T 312ba1fe180943208a291beb61757cdc P 52b4fd0e03f44784a1a308be185e8967: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260619 17:00:54.131336 2177 raft_consensus.cc:348] T 312ba1fe180943208a291beb61757cdc P ef2f3094aed44ce1b1eb1db5cd7065f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2f3094aed44ce1b1eb1db5cd7065f6" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 42009 } } peers { permanent_uuid: "42df110a75084262aa072a23bd7b768e" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34947 } } peers { permanent_uuid: "52b4fd0e03f44784a1a308be185e8967" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35425 } }
I20260619 17:00:54.131659 2177 raft_consensus.cc:374] T 312ba1fe180943208a291beb61757cdc P ef2f3094aed44ce1b1eb1db5cd7065f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:00:54.131743 2177 raft_consensus.cc:729] T 312ba1fe180943208a291beb61757cdc P ef2f3094aed44ce1b1eb1db5cd7065f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef2f3094aed44ce1b1eb1db5cd7065f6, State: Initialized, Role: FOLLOWER
I20260619 17:00:54.131963 2177 consensus_queue.cc:260] T 312ba1fe180943208a291beb61757cdc P ef2f3094aed44ce1b1eb1db5cd7065f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2f3094aed44ce1b1eb1db5cd7065f6" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 42009 } } peers { permanent_uuid: "42df110a75084262aa072a23bd7b768e" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34947 } } peers { permanent_uuid: "52b4fd0e03f44784a1a308be185e8967" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35425 } }
I20260619 17:00:54.132330 2177 ts_tablet_manager.cc:1428] T 312ba1fe180943208a291beb61757cdc P ef2f3094aed44ce1b1eb1db5cd7065f6: Time spent starting tablet: real 0.004s user 0.001s sys 0.003s
I20260619 17:00:54.132491 2144 heartbeater.cc:503] Master 127.20.231.190:41497 was elected leader, sending a full tablet report...
I20260619 17:00:54.133460 2178 raft_consensus.cc:348] T 312ba1fe180943208a291beb61757cdc P 52b4fd0e03f44784a1a308be185e8967 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2f3094aed44ce1b1eb1db5cd7065f6" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 42009 } } peers { permanent_uuid: "42df110a75084262aa072a23bd7b768e" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34947 } } peers { permanent_uuid: "52b4fd0e03f44784a1a308be185e8967" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35425 } }
I20260619 17:00:54.133636 2178 raft_consensus.cc:374] T 312ba1fe180943208a291beb61757cdc P 52b4fd0e03f44784a1a308be185e8967 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:00:54.133679 2178 raft_consensus.cc:729] T 312ba1fe180943208a291beb61757cdc P 52b4fd0e03f44784a1a308be185e8967 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52b4fd0e03f44784a1a308be185e8967, State: Initialized, Role: FOLLOWER
I20260619 17:00:54.133819 2178 consensus_queue.cc:260] T 312ba1fe180943208a291beb61757cdc P 52b4fd0e03f44784a1a308be185e8967 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2f3094aed44ce1b1eb1db5cd7065f6" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 42009 } } peers { permanent_uuid: "42df110a75084262aa072a23bd7b768e" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34947 } } peers { permanent_uuid: "52b4fd0e03f44784a1a308be185e8967" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35425 } }
I20260619 17:00:54.134111 2178 ts_tablet_manager.cc:1428] T 312ba1fe180943208a291beb61757cdc P 52b4fd0e03f44784a1a308be185e8967: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260619 17:00:54.134267 1876 heartbeater.cc:503] Master 127.20.231.190:41497 was elected leader, sending a full tablet report...
I20260619 17:00:54.134925 2179 tablet_bootstrap.cc:492] T 312ba1fe180943208a291beb61757cdc P 42df110a75084262aa072a23bd7b768e: No bootstrap required, opened a new log
I20260619 17:00:54.135039 2179 ts_tablet_manager.cc:1397] T 312ba1fe180943208a291beb61757cdc P 42df110a75084262aa072a23bd7b768e: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260619 17:00:54.137527 2179 raft_consensus.cc:348] T 312ba1fe180943208a291beb61757cdc P 42df110a75084262aa072a23bd7b768e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2f3094aed44ce1b1eb1db5cd7065f6" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 42009 } } peers { permanent_uuid: "42df110a75084262aa072a23bd7b768e" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34947 } } peers { permanent_uuid: "52b4fd0e03f44784a1a308be185e8967" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35425 } }
I20260619 17:00:54.137687 2179 raft_consensus.cc:374] T 312ba1fe180943208a291beb61757cdc P 42df110a75084262aa072a23bd7b768e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:00:54.137728 2179 raft_consensus.cc:729] T 312ba1fe180943208a291beb61757cdc P 42df110a75084262aa072a23bd7b768e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42df110a75084262aa072a23bd7b768e, State: Initialized, Role: FOLLOWER
I20260619 17:00:54.137962 2179 consensus_queue.cc:260] T 312ba1fe180943208a291beb61757cdc P 42df110a75084262aa072a23bd7b768e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2f3094aed44ce1b1eb1db5cd7065f6" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 42009 } } peers { permanent_uuid: "42df110a75084262aa072a23bd7b768e" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34947 } } peers { permanent_uuid: "52b4fd0e03f44784a1a308be185e8967" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35425 } }
I20260619 17:00:54.138284 2179 ts_tablet_manager.cc:1428] T 312ba1fe180943208a291beb61757cdc P 42df110a75084262aa072a23bd7b768e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 17:00:54.138437 2010 heartbeater.cc:503] Master 127.20.231.190:41497 was elected leader, sending a full tablet report...
W20260619 17:00:54.172562 1877 tablet.cc:2369] T 312ba1fe180943208a291beb61757cdc P 52b4fd0e03f44784a1a308be185e8967: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 17:00:54.195348 2184 raft_consensus.cc:482] T 312ba1fe180943208a291beb61757cdc P 52b4fd0e03f44784a1a308be185e8967 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 17:00:54.195529 2184 raft_consensus.cc:504] T 312ba1fe180943208a291beb61757cdc P 52b4fd0e03f44784a1a308be185e8967 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2f3094aed44ce1b1eb1db5cd7065f6" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 42009 } } peers { permanent_uuid: "42df110a75084262aa072a23bd7b768e" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34947 } } peers { permanent_uuid: "52b4fd0e03f44784a1a308be185e8967" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35425 } }
I20260619 17:00:54.196164 2184 leader_election.cc:288] T 312ba1fe180943208a291beb61757cdc P 52b4fd0e03f44784a1a308be185e8967 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef2f3094aed44ce1b1eb1db5cd7065f6 (127.20.231.131:42009), 42df110a75084262aa072a23bd7b768e (127.20.231.130:34947)
I20260619 17:00:54.200973 2176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:54.196790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:51939 (local address 127.20.231.130:34947)
0619 17:00:54.196832 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:54.196835 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:54.196946 (+ 111us) server_negotiation.cc:407] Connection header received
0619 17:00:54.197094 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:54.197095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:54.197106 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:54.197160 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:00:54.197403 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:54.198136 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:54.199856 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:54.200498 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:54.200553 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:54.200562 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:54.200840 (+ 278us) server_negotiation.cc:299] Negotiation successful
0619 17:00:54.200887 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 17:00:54.200973 2175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:54.196467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:51447 (local address 127.20.231.131:42009)
0619 17:00:54.196514 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:54.196517 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:54.196680 (+ 163us) server_negotiation.cc:407] Connection header received
0619 17:00:54.196825 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:54.196827 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:54.196840 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:54.196877 (+ 37us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:00:54.197195 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:54.198072 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:54.199856 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:54.200498 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:54.200553 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:54.200562 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:54.200840 (+ 278us) server_negotiation.cc:299] Negotiation successful
0619 17:00:54.200887 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 17:00:54.201305 2187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:54.196601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:34947 (local address 127.20.231.129:51939)
0619 17:00:54.196886 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:54.196897 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 17:00:54.196969 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:00:54.197256 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:00:54.197256 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:00:54.197258 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:00:54.197316 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:54.197317 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:54.198227 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:54.198227 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:54.199728 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:54.199731 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:54.200655 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:54.200656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:54.200741 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:54.200747 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 17:00:54.201112 (+ 365us) client_negotiation.cc:240] Negotiation successful
0619 17:00:54.201177 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":11,"thread_start_us":69,"threads_started":1}
I20260619 17:00:54.201694 2186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:54.196367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:42009 (local address 127.20.231.129:51447)
0619 17:00:54.196606 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:54.196619 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 17:00:54.196713 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:00:54.196967 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:00:54.196968 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:00:54.196970 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:00:54.197104 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:54.197107 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:54.198209 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:54.198210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:54.199728 (+ 1518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:54.199738 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:54.200655 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:54.200656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:54.200750 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:54.200752 (+ 2us) client_negotiation.cc:769] Sending connection context
0619 17:00:54.201557 (+ 805us) client_negotiation.cc:240] Negotiation successful
0619 17:00:54.201608 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"mutex_wait_us":122,"thread_start_us":117,"threads_started":1}
I20260619 17:00:54.201679 1964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "312ba1fe180943208a291beb61757cdc" candidate_uuid: "52b4fd0e03f44784a1a308be185e8967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42df110a75084262aa072a23bd7b768e" is_pre_election: true
I20260619 17:00:54.201870 1964 raft_consensus.cc:2454] T 312ba1fe180943208a291beb61757cdc P 42df110a75084262aa072a23bd7b768e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b4fd0e03f44784a1a308be185e8967 in term 0.
I20260619 17:00:54.202078 2098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "312ba1fe180943208a291beb61757cdc" candidate_uuid: "52b4fd0e03f44784a1a308be185e8967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2f3094aed44ce1b1eb1db5cd7065f6" is_pre_election: true
I20260619 17:00:54.202143 1763 leader_election.cc:302] T 312ba1fe180943208a291beb61757cdc P 52b4fd0e03f44784a1a308be185e8967 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 42df110a75084262aa072a23bd7b768e, 52b4fd0e03f44784a1a308be185e8967; no voters:
I20260619 17:00:54.202270 2098 raft_consensus.cc:2454] T 312ba1fe180943208a291beb61757cdc P ef2f3094aed44ce1b1eb1db5cd7065f6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 52b4fd0e03f44784a1a308be185e8967 in term 0.
I20260619 17:00:54.202301 2184 raft_consensus.cc:2781] T 312ba1fe180943208a291beb61757cdc P 52b4fd0e03f44784a1a308be185e8967 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 17:00:54.202354 2184 raft_consensus.cc:482] T 312ba1fe180943208a291beb61757cdc P 52b4fd0e03f44784a1a308be185e8967 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 17:00:54.202381 2184 raft_consensus.cc:3037] T 312ba1fe180943208a291beb61757cdc P 52b4fd0e03f44784a1a308be185e8967 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:00:54.203274 2184 raft_consensus.cc:504] T 312ba1fe180943208a291beb61757cdc P 52b4fd0e03f44784a1a308be185e8967 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2f3094aed44ce1b1eb1db5cd7065f6" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 42009 } } peers { permanent_uuid: "42df110a75084262aa072a23bd7b768e" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34947 } } peers { permanent_uuid: "52b4fd0e03f44784a1a308be185e8967" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35425 } }
I20260619 17:00:54.203420 2184 leader_election.cc:288] T 312ba1fe180943208a291beb61757cdc P 52b4fd0e03f44784a1a308be185e8967 [CANDIDATE]: Term 1 election: Requested vote from peers ef2f3094aed44ce1b1eb1db5cd7065f6 (127.20.231.131:42009), 42df110a75084262aa072a23bd7b768e (127.20.231.130:34947)
I20260619 17:00:54.203612 1964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "312ba1fe180943208a291beb61757cdc" candidate_uuid: "52b4fd0e03f44784a1a308be185e8967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42df110a75084262aa072a23bd7b768e"
I20260619 17:00:54.203660 2098 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "312ba1fe180943208a291beb61757cdc" candidate_uuid: "52b4fd0e03f44784a1a308be185e8967" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2f3094aed44ce1b1eb1db5cd7065f6"
I20260619 17:00:54.203719 1964 raft_consensus.cc:3037] T 312ba1fe180943208a291beb61757cdc P 42df110a75084262aa072a23bd7b768e [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:00:54.203791 2098 raft_consensus.cc:3037] T 312ba1fe180943208a291beb61757cdc P ef2f3094aed44ce1b1eb1db5cd7065f6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:00:54.204537 2098 raft_consensus.cc:2454] T 312ba1fe180943208a291beb61757cdc P ef2f3094aed44ce1b1eb1db5cd7065f6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b4fd0e03f44784a1a308be185e8967 in term 1.
I20260619 17:00:54.204605 1964 raft_consensus.cc:2454] T 312ba1fe180943208a291beb61757cdc P 42df110a75084262aa072a23bd7b768e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 52b4fd0e03f44784a1a308be185e8967 in term 1.
I20260619 17:00:54.204761 1763 leader_election.cc:302] T 312ba1fe180943208a291beb61757cdc P 52b4fd0e03f44784a1a308be185e8967 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 52b4fd0e03f44784a1a308be185e8967, ef2f3094aed44ce1b1eb1db5cd7065f6; no voters:
I20260619 17:00:54.204885 2184 raft_consensus.cc:2781] T 312ba1fe180943208a291beb61757cdc P 52b4fd0e03f44784a1a308be185e8967 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:00:54.205108 2184 raft_consensus.cc:686] T 312ba1fe180943208a291beb61757cdc P 52b4fd0e03f44784a1a308be185e8967 [term 1 LEADER]: Becoming Leader. State: Replica: 52b4fd0e03f44784a1a308be185e8967, State: Running, Role: LEADER
I20260619 17:00:54.205240 2184 consensus_queue.cc:237] T 312ba1fe180943208a291beb61757cdc P 52b4fd0e03f44784a1a308be185e8967 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2f3094aed44ce1b1eb1db5cd7065f6" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 42009 } } peers { permanent_uuid: "42df110a75084262aa072a23bd7b768e" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34947 } } peers { permanent_uuid: "52b4fd0e03f44784a1a308be185e8967" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35425 } }
I20260619 17:00:54.206140 1678 catalog_manager.cc:5495] T 312ba1fe180943208a291beb61757cdc P 52b4fd0e03f44784a1a308be185e8967 reported cstate change: term changed from 0 to 1, leader changed from <none> to 52b4fd0e03f44784a1a308be185e8967 (127.20.231.129). New cstate: current_term: 1 leader_uuid: "52b4fd0e03f44784a1a308be185e8967" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef2f3094aed44ce1b1eb1db5cd7065f6" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 42009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42df110a75084262aa072a23bd7b768e" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "52b4fd0e03f44784a1a308be185e8967" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35425 } health_report { overall_health: HEALTHY } } }
I20260619 17:00:54.213876 2174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:54.211451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37466 (local address 127.20.231.129:35425)
0619 17:00:54.211507 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:54.211509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:54.211516 (+ 7us) server_negotiation.cc:407] Connection header received
0619 17:00:54.211546 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:54.211547 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:54.211571 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:54.211649 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 17:00:54.211810 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:54.212279 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:54.213042 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:54.213273 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:54.213362 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:54.213542 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 17:00:54.213651 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 17:00:54.213743 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:54.213767 (+ 24us) server_negotiation.cc:299] Negotiation successful
0619 17:00:54.213787 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 17:00:54.215692 1964 raft_consensus.cc:1261] T 312ba1fe180943208a291beb61757cdc P 42df110a75084262aa072a23bd7b768e [term 1 FOLLOWER]: Refusing update from remote peer 52b4fd0e03f44784a1a308be185e8967: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 17:00:54.215693 2098 raft_consensus.cc:1261] T 312ba1fe180943208a291beb61757cdc P ef2f3094aed44ce1b1eb1db5cd7065f6 [term 1 FOLLOWER]: Refusing update from remote peer 52b4fd0e03f44784a1a308be185e8967: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 17:00:54.216077 2188 consensus_queue.cc:1035] T 312ba1fe180943208a291beb61757cdc P 52b4fd0e03f44784a1a308be185e8967 [LEADER]: Connected to new peer: Peer: permanent_uuid: "42df110a75084262aa072a23bd7b768e" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 17:00:54.216307 2184 consensus_queue.cc:1035] T 312ba1fe180943208a291beb61757cdc P 52b4fd0e03f44784a1a308be185e8967 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2f3094aed44ce1b1eb1db5cd7065f6" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 42009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 17:00:54.219071 2191 mvcc.cc:205] Tried to move back new op lower bound from 7298615108464857088 to 7298615108425506816. Current Snapshot: MvccSnapshot[applied={T|T < 7298615108464857088}]
I20260619 17:00:54.222993 1677 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60784:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 17:00:54.223115 1677 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60784:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 17:00:54.223326 2193 mvcc.cc:205] Tried to move back new op lower bound from 7298615108464857088 to 7298615108425506816. Current Snapshot: MvccSnapshot[applied={T|T < 7298615108464857088}]
I20260619 17:00:54.223857 1677 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4f4212d9747442968b6f74a2fb4de929: Sending DeleteTablet for 3 replicas of tablet 312ba1fe180943208a291beb61757cdc
I20260619 17:00:54.224284 2078 tablet_service.cc:1514] Processing DeleteTablet for tablet 312ba1fe180943208a291beb61757cdc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:00:54 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:52102
I20260619 17:00:54.224313 1792 tablet_service.cc:1514] Processing DeleteTablet for tablet 312ba1fe180943208a291beb61757cdc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:00:54 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:37452
I20260619 17:00:54.224555 1944 tablet_service.cc:1514] Processing DeleteTablet for tablet 312ba1fe180943208a291beb61757cdc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:00:54 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:46236
I20260619 17:00:54.224612 2200 tablet_replica.cc:330] T 312ba1fe180943208a291beb61757cdc P ef2f3094aed44ce1b1eb1db5cd7065f6: stopping tablet replica
I20260619 17:00:54.224939 2200 raft_consensus.cc:2229] T 312ba1fe180943208a291beb61757cdc P ef2f3094aed44ce1b1eb1db5cd7065f6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 17:00:54.225062 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 1716
I20260619 17:00:54.225085 2200 raft_consensus.cc:2258] T 312ba1fe180943208a291beb61757cdc P ef2f3094aed44ce1b1eb1db5cd7065f6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 17:00:54.226673 2201 tablet_replica.cc:330] T 312ba1fe180943208a291beb61757cdc P 42df110a75084262aa072a23bd7b768e: stopping tablet replica
I20260619 17:00:54.226799 2201 raft_consensus.cc:2229] T 312ba1fe180943208a291beb61757cdc P 42df110a75084262aa072a23bd7b768e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 17:00:54.226974 2201 raft_consensus.cc:2258] T 312ba1fe180943208a291beb61757cdc P 42df110a75084262aa072a23bd7b768e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 17:00:54.227372 2201 ts_tablet_manager.cc:1905] T 312ba1fe180943208a291beb61757cdc P 42df110a75084262aa072a23bd7b768e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 17:00:54.229015 2201 ts_tablet_manager.cc:1918] T 312ba1fe180943208a291beb61757cdc P 42df110a75084262aa072a23bd7b768e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 17:00:54.229104 2201 log.cc:1196] T 312ba1fe180943208a291beb61757cdc P 42df110a75084262aa072a23bd7b768e: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-1/wal/wals/312ba1fe180943208a291beb61757cdc
I20260619 17:00:54.229362 2201 ts_tablet_manager.cc:1939] T 312ba1fe180943208a291beb61757cdc P 42df110a75084262aa072a23bd7b768e: Deleting consensus metadata
I20260619 17:00:54.229797 1662 catalog_manager.cc:4841] TS 42df110a75084262aa072a23bd7b768e (127.20.231.130:34947): tablet 312ba1fe180943208a291beb61757cdc (table test-table [id=3ea7361a9772465cbc049003e0853f4b]) successfully deleted
W20260619 17:00:54.230412 1665 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.231.129:35425 (error 108)
W20260619 17:00:54.231801 2172 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 17:00:54.230838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:35425 (local address 127.0.0.1:37474)
0619 17:00:54.231031 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:54.231042 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 17:00:54.231077 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:00:54.231755 (+ 678us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.129:35425: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":171}
W20260619 17:00:54.232017 1665 catalog_manager.cc:4584] TS 52b4fd0e03f44784a1a308be185e8967 (127.20.231.129:35425): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 312ba1fe180943208a291beb61757cdc: Network error: Client connection negotiation failed: client connection to 127.20.231.129:35425: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 17:00:54.232191 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 1880
I20260619 17:00:54.235086 2200 ts_tablet_manager.cc:1905] T 312ba1fe180943208a291beb61757cdc P ef2f3094aed44ce1b1eb1db5cd7065f6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 17:00:54.237047 2200 ts_tablet_manager.cc:1918] T 312ba1fe180943208a291beb61757cdc P ef2f3094aed44ce1b1eb1db5cd7065f6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 17:00:54.237130 2200 log.cc:1196] T 312ba1fe180943208a291beb61757cdc P ef2f3094aed44ce1b1eb1db5cd7065f6: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781888301345144-21406-0/minicluster-data/ts-2/wal/wals/312ba1fe180943208a291beb61757cdc
I20260619 17:00:54.237490 2200 ts_tablet_manager.cc:1939] T 312ba1fe180943208a291beb61757cdc P ef2f3094aed44ce1b1eb1db5cd7065f6: Deleting consensus metadata
I20260619 17:00:54.238008 1662 catalog_manager.cc:4841] TS ef2f3094aed44ce1b1eb1db5cd7065f6 (127.20.231.131:42009): tablet 312ba1fe180943208a291beb61757cdc (table test-table [id=3ea7361a9772465cbc049003e0853f4b]) successfully deleted
I20260619 17:00:54.238778 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 2013
I20260619 17:00:54.245309 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 1645
I20260619 17:00:54.251037 21406 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 17:00:54.596107 21406 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 17:00:54.596230 21406 mini_ranger.cc:66] Stopping Ranger...
I20260619 17:00:54.941548 21406 mini_ranger.cc:68] Stopped Ranger
2026-06-19 17:00:54.941 UTC [711] LOG: received smart shutdown request
2026-06-19 17:00:54.946 UTC [711] LOG: background worker "logical replication launcher" (PID 720) exited with exit code 1
2026-06-19 17:00:54.946 UTC [715] LOG: shutting down
2026-06-19 17:00:55.143 UTC [711] LOG: database system is shut down
2026-06-19T17:00:55Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50111 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 17:00:55 dist-test-slave-vnbj krb5kdc[2212](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 17:00:55 dist-test-slave-vnbj krb5kdc[2212](info): set up 2 sockets
Jun 19 17:00:55 dist-test-slave-vnbj krb5kdc[2212](info): commencing operation
krb5kdc: starting...
W20260619 17:00:56.342751 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 17:00:56 dist-test-slave-vnbj krb5kdc[2212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T17:00:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T17:00:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.231.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
I20260619 17:00:56.482388 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:40615
--webserver_interface=127.20.231.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:46673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:40615,127.20.231.189:36557,127.20.231.188:36683
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:00:56.564054 2225 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:00:56.564285 2225 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:00:56.564308 2225 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:00:56.565465 2225 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:00:56.565505 2225 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:00:56.565518 2225 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:00:56.565531 2225 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:00:56.565543 2225 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:00:56.567085 2225 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:46673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:40615,127.20.231.189:36557,127.20.231.188:36683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:40615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2225
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:00:56.567291 2225 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:00:56.567471 2225 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:00:56.572690 2230 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:00:56.572719 2231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:00:56.572798 2225 server_base.cc:1034] running on GCE node
W20260619 17:00:56.572697 2233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:00:56.573154 2225 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:00:56.573493 2225 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:00:56.574651 2225 hybrid_clock.cc:648] HybridClock initialized: now 1781888456574630 us; error 42 us; skew 500 ppm
Jun 19 17:00:56 dist-test-slave-vnbj krb5kdc[2212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888456, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:00:56.577381 2225 init.cc:377] Logged in from keytab as kudu/127.20.231.190@KRBTEST.COM (short username kudu)
I20260619 17:00:56.577847 2225 webserver.cc:460] Webserver started at http://127.20.231.190:34881/ using document root <none> and password file <none>
I20260619 17:00:56.578079 2225 fs_manager.cc:359] Metadata directory not provided
I20260619 17:00:56.578125 2225 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:00:56.578250 2225 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:00:56.579427 2225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "3aaa4258be0740ed956f4dc48f232e4d"
format_stamp: "Formatted at 2026-06-19 17:00:56 on dist-test-slave-vnbj"
server_key: "5897c2fa15b0a797eb31cb637f44b545"
server_key_iv: "b83f55ba387d6611b0af7515d2d9eb0e"
server_key_version: "encryptionkey@0"
I20260619 17:00:56.579792 2225 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "3aaa4258be0740ed956f4dc48f232e4d"
format_stamp: "Formatted at 2026-06-19 17:00:56 on dist-test-slave-vnbj"
server_key: "5897c2fa15b0a797eb31cb637f44b545"
server_key_iv: "b83f55ba387d6611b0af7515d2d9eb0e"
server_key_version: "encryptionkey@0"
I20260619 17:00:56.581163 2225 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 17:00:56.582069 2240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:00:56.582299 2225 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:00:56.582391 2225 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "3aaa4258be0740ed956f4dc48f232e4d"
format_stamp: "Formatted at 2026-06-19 17:00:56 on dist-test-slave-vnbj"
server_key: "5897c2fa15b0a797eb31cb637f44b545"
server_key_iv: "b83f55ba387d6611b0af7515d2d9eb0e"
server_key_version: "encryptionkey@0"
I20260619 17:00:56.582448 2225 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:00:56.599069 2225 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:00:56.601275 2225 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:00:56.601409 2225 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:00:56.605432 2225 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:40615
I20260619 17:00:56.605486 2292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:40615 every 8 connection(s)
I20260619 17:00:56.605845 2225 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 17:00:56.606721 2293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:00:56.607661 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 2225
I20260619 17:00:56.607757 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 17:00:56.607954 21406 external_mini_cluster.cc:1428] Setting key 72bde8d03f9a8dbdc11be149556e9f6f
I20260619 17:00:56.608471 2293 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.231.190" port: 40615 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 17:00:56 dist-test-slave-vnbj krb5kdc[2212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888456, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.231.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.189@KRBTEST.COM" created.
I20260619 17:00:56.619580 2297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.609223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:40615 (local address 127.0.0.1:37092)
0619 17:00:56.609393 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:56.609441 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 17:00:56.609894 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:00:56.610212 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:00:56.610215 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:00:56.610516 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:00:56.610682 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:56.610686 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.611776 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.611777 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:56.613004 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:56.613008 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.613506 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.613507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:56.613601 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:56.614299 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:00:56.614312 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:00:56.616049 (+ 1737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:00:56.617913 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:00:56.617918 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:00:56.617920 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:00:56.618106 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:00:56.618433 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:00:56.618434 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:00:56.618434 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:00:56.618526 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:00:56.618959 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:00:56.618964 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:00:56.619029 (+ 65us) client_negotiation.cc:769] Sending connection context
0619 17:00:56.619145 (+ 116us) client_negotiation.cc:240] Negotiation successful
0619 17:00:56.619368 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":97,"thread_start_us":89,"threads_started":1}
I20260619 17:00:56.619580 2296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.609232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37092 (local address 127.20.231.190:40615)
0619 17:00:56.609496 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:56.609500 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:56.609535 (+ 35us) server_negotiation.cc:407] Connection header received
0619 17:00:56.610057 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:56.610060 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:56.610082 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:56.610197 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:00:56.610774 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.611667 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.613098 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.613413 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.613521 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:56.616154 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:00:56.616161 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:00:56.616162 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:00:56.616169 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:00:56.617812 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:56.618232 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:56.618236 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:56.618239 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:56.618350 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:56.618591 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:56.618591 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:56.618592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:56.618900 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:00:56.618981 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:56.619170 (+ 189us) server_negotiation.cc:299] Negotiation successful
0619 17:00:56.619323 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":231,"spinlock_wait_cycles":8064,"thread_start_us":91,"threads_started":1}
I20260619 17:00:56.620321 2293 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.231.189" port: 36557 } has no permanent_uuid. Determining permanent_uuid...
I20260619 17:00:56.620705 2297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.620587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.189:36557 (local address 127.0.0.1:50298)
0619 17:00:56.620635 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:56.620678 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.189:36557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 17:00:56.620837 2244 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.231.189:36557: connect: Connection refused (error 111)
I20260619 17:00:56.621286 2297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.621163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.189:36557 (local address 127.0.0.1:50306)
0619 17:00:56.621212 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:56.621253 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.189:36557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 17:00:56.621405 2293 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.231.189:36557: Network error: Client connection negotiation failed: client connection to 127.20.231.189:36557: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.189.keytab.
Entry for principal kudu/127.20.231.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.189.keytab.
I20260619 17:00:56.634001 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.189:36557
--webserver_interface=127.20.231.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.189.keytab
--principal=kudu/127.20.231.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:46673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:40615,127.20.231.189:36557,127.20.231.188:36683
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 17:00:56.670634 2293 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.231.189" port: 36557 } attempt: 1
I20260619 17:00:56.671305 2297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.671091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.189:36557 (local address 127.0.0.1:50316)
0619 17:00:56.671217 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:56.671268 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.189:36557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260619 17:00:56.671828 2297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.671703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.189:36557 (local address 127.0.0.1:50332)
0619 17:00:56.671769 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:56.671802 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.189:36557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260619 17:00:56.671964 2293 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.231.189:36557: Network error: Client connection negotiation failed: client connection to 127.20.231.189:36557: connect: Connection refused (error 111)
W20260619 17:00:56.719429 2300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:00:56.719677 2300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:00:56.719705 2300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:00:56.721012 2300 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:00:56.721069 2300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:00:56.721087 2300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:00:56.721100 2300 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:00:56.721112 2300 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:00:56.722745 2300 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:46673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:40615,127.20.231.189:36557,127.20.231.188:36683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.189.keytab
--principal=kudu/127.20.231.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.189:36557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.231.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:00:56.723045 2300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:00:56.723331 2300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 17:00:56.728559 2300 server_base.cc:1034] running on GCE node
W20260619 17:00:56.728538 2308 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:00:56.728524 2305 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:00:56.728518 2306 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:00:56.729084 2300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:00:56.729429 2300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:00:56.730615 2300 hybrid_clock.cc:648] HybridClock initialized: now 1781888456730588 us; error 42 us; skew 500 ppm
Jun 19 17:00:56 dist-test-slave-vnbj krb5kdc[2212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888456, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:00:56.733312 2300 init.cc:377] Logged in from keytab as kudu/127.20.231.189@KRBTEST.COM (short username kudu)
I20260619 17:00:56.733814 2300 webserver.cc:460] Webserver started at http://127.20.231.189:38815/ using document root <none> and password file <none>
I20260619 17:00:56.734063 2300 fs_manager.cc:359] Metadata directory not provided
I20260619 17:00:56.734110 2300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:00:56.734282 2300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:00:56.735458 2300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-1/data/instance:
uuid: "854fd43d6785425fac06d05c4fd6923e"
format_stamp: "Formatted at 2026-06-19 17:00:56 on dist-test-slave-vnbj"
server_key: "88fc9cf9b4a34628b5e3820fbc10a015"
server_key_iv: "566fb9cae38936ea4d3177ae62c847f4"
server_key_version: "encryptionkey@0"
I20260619 17:00:56.735827 2300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-1/wal/instance:
uuid: "854fd43d6785425fac06d05c4fd6923e"
format_stamp: "Formatted at 2026-06-19 17:00:56 on dist-test-slave-vnbj"
server_key: "88fc9cf9b4a34628b5e3820fbc10a015"
server_key_iv: "566fb9cae38936ea4d3177ae62c847f4"
server_key_version: "encryptionkey@0"
I20260619 17:00:56.737205 2300 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 17:00:56.738103 2315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:00:56.738307 2300 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:00:56.738442 2300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-1/wal
uuid: "854fd43d6785425fac06d05c4fd6923e"
format_stamp: "Formatted at 2026-06-19 17:00:56 on dist-test-slave-vnbj"
server_key: "88fc9cf9b4a34628b5e3820fbc10a015"
server_key_iv: "566fb9cae38936ea4d3177ae62c847f4"
server_key_version: "encryptionkey@0"
I20260619 17:00:56.738525 2300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:00:56.740144 2293 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.231.189" port: 36557 } attempt: 2
I20260619 17:00:56.740588 2297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.740454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.189:36557 (local address 127.0.0.1:50344)
0619 17:00:56.740518 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:56.740559 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.189:36557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 17:00:56.740960 2297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.740859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.189:36557 (local address 127.0.0.1:50346)
0619 17:00:56.740907 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:56.740942 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.189:36557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 17:00:56.741052 2293 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.231.189:36557: Network error: Client connection negotiation failed: client connection to 127.20.231.189:36557: connect: Connection refused (error 111)
I20260619 17:00:56.747604 2300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:00:56.750002 2300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:00:56.750207 2300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:00:56.755226 2300 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.189:36557
I20260619 17:00:56.755314 2367 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.189:36557 every 8 connection(s)
I20260619 17:00:56.755730 2300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-1/data/info.pb
I20260619 17:00:56.756711 2368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:00:56.758481 2368 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.231.190" port: 40615 } has no permanent_uuid. Determining permanent_uuid...
I20260619 17:00:56.759393 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 2300
I20260619 17:00:56.759517 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-1/wal/instance
I20260619 17:00:56.759734 21406 external_mini_cluster.cc:1428] Setting key a2d6b6d39e896c029fc9a825963a8a3f
Jun 19 17:00:56 dist-test-slave-vnbj krb5kdc[2212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888456, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.189@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.231.188@KRBTEST.COM; defaulting to no policy
I20260619 17:00:56.769479 2296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.759378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37096 (local address 127.20.231.190:40615)
0619 17:00:56.759443 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:56.759446 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:56.759745 (+ 299us) server_negotiation.cc:407] Connection header received
0619 17:00:56.760303 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:56.760304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:56.760321 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:56.760387 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:00:56.761072 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.761566 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.762573 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.762934 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.763004 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:56.765715 (+ 2711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:00:56.765717 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:00:56.765718 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:00:56.765727 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:00:56.767681 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:56.768125 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:56.768127 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:56.768127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:56.768189 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:56.768629 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:56.768631 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:56.768631 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:56.768802 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:00:56.768885 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:56.769266 (+ 381us) server_negotiation.cc:299] Negotiation successful
0619 17:00:56.769360 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 17:00:56.769719 2370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.759330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:40615 (local address 127.0.0.1:37096)
0619 17:00:56.759597 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:56.759649 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 17:00:56.760189 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:00:56.760470 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:00:56.760473 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:00:56.760761 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:00:56.760968 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:56.760972 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.761679 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.761680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:56.762473 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:56.762475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.763026 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.763027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:56.763164 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:56.763836 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:00:56.763848 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:00:56.765607 (+ 1759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:00:56.767789 (+ 2182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:00:56.767795 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:00:56.767798 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:00:56.768039 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:00:56.768295 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:00:56.768296 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:00:56.768297 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:00:56.768470 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:00:56.768891 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:00:56.768899 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:00:56.769102 (+ 203us) client_negotiation.cc:769] Sending connection context
0619 17:00:56.769245 (+ 143us) client_negotiation.cc:240] Negotiation successful
0619 17:00:56.769450 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":90,"threads_started":1}
I20260619 17:00:56.770305 2368 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.231.189" port: 36557 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.188@KRBTEST.COM" created.
Jun 19 17:00:56 dist-test-slave-vnbj krb5kdc[2212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888456, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.189@KRBTEST.COM for kudu/127.20.231.189@KRBTEST.COM
I20260619 17:00:56.779683 2370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.770603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.189:36557 (local address 127.0.0.1:50362)
0619 17:00:56.770786 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:56.770796 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 17:00:56.770840 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:00:56.771059 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:00:56.771060 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:00:56.771187 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:00:56.771265 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:56.771267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.772529 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.772530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:56.773108 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:56.773110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.773815 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.773816 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:56.773882 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:56.774447 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:00:56.774453 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:00:56.776052 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:00:56.778299 (+ 2247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:00:56.778300 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:00:56.778301 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:00:56.778498 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:00:56.778839 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:00:56.778840 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:00:56.778840 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:00:56.778981 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:00:56.779407 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:00:56.779408 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:00:56.779454 (+ 46us) client_negotiation.cc:769] Sending connection context
0619 17:00:56.779521 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 17:00:56.779606 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170}
I20260619 17:00:56.779804 2372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.770664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50362 (local address 127.20.231.189:36557)
0619 17:00:56.770803 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:56.770807 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:56.770837 (+ 30us) server_negotiation.cc:407] Connection header received
0619 17:00:56.770949 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:56.770952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:56.770966 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:56.771035 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:00:56.771326 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.772415 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.773197 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.773698 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.773780 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:56.776146 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:00:56.776156 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:00:56.776157 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:00:56.776165 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:00:56.778161 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:56.778633 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:56.778637 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:56.778638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:56.778707 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:56.779072 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:56.779073 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:56.779073 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:56.779306 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:00:56.779404 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:56.779605 (+ 201us) server_negotiation.cc:299] Negotiation successful
0619 17:00:56.779693 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":80,"threads_started":1}
I20260619 17:00:56.780505 2368 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.231.188" port: 36683 } has no permanent_uuid. Determining permanent_uuid...
I20260619 17:00:56.780965 2370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.780845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:36683 (local address 127.0.0.1:40546)
0619 17:00:56.780895 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:56.780945 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.188:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260619 17:00:56.781116 2318 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.231.188:36683: connect: Connection refused (error 111)
I20260619 17:00:56.781601 2370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.781446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:36683 (local address 127.0.0.1:40554)
0619 17:00:56.781536 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:56.781571 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.188:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260619 17:00:56.781814 2368 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.231.188:36683: Network error: Client connection negotiation failed: client connection to 127.20.231.188:36683: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.188.keytab.
Entry for principal kudu/127.20.231.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.188.keytab.
I20260619 17:00:56.789970 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.188:36683
--webserver_interface=127.20.231.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.188.keytab
--principal=kudu/127.20.231.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:46673
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:40615,127.20.231.189:36557,127.20.231.188:36683
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 17:00:56.831058 2368 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.231.188" port: 36683 } attempt: 1
I20260619 17:00:56.831691 2370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.831543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:36683 (local address 127.0.0.1:40564)
0619 17:00:56.831616 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:56.831660 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.188:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260619 17:00:56.832101 2370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.832000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:36683 (local address 127.0.0.1:40566)
0619 17:00:56.832051 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:56.832083 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.188:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 17:00:56.832254 2368 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.231.188:36683: Network error: Client connection negotiation failed: client connection to 127.20.231.188:36683: connect: Connection refused (error 111)
I20260619 17:00:56.832235 2293 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.231.189" port: 36557 } attempt: 3
Jun 19 17:00:56 dist-test-slave-vnbj krb5kdc[2212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888456, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for kudu/127.20.231.189@KRBTEST.COM
I20260619 17:00:56.839605 2372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.832570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50364 (local address 127.20.231.189:36557)
0619 17:00:56.832611 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:56.832613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:56.832625 (+ 12us) server_negotiation.cc:407] Connection header received
0619 17:00:56.832693 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:56.832695 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:56.832707 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:56.832756 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:00:56.833075 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.833601 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.834305 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.834540 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.834617 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:56.836806 (+ 2189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:00:56.836807 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:00:56.836807 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:00:56.836816 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:00:56.838334 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:56.838733 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:56.838734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:56.838735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:56.838794 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:56.839040 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:56.839041 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:56.839042 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:56.839167 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:00:56.839249 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:56.839421 (+ 172us) server_negotiation.cc:299] Negotiation successful
0619 17:00:56.839502 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 17:00:56.839613 2297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.832530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.189:36557 (local address 127.0.0.1:50364)
0619 17:00:56.832568 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:56.832579 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 17:00:56.832629 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:00:56.832758 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:00:56.832759 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:00:56.832926 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:00:56.832988 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:56.832990 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.833695 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.833704 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:56.834220 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:56.834222 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.834615 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.834616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:56.834702 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:56.835226 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:00:56.835231 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:00:56.836633 (+ 1402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:00:56.838444 (+ 1811us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:00:56.838445 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:00:56.838446 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:00:56.838633 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:00:56.838900 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:00:56.838900 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:00:56.838901 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:00:56.838966 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:00:56.839240 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:00:56.839241 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:00:56.839294 (+ 53us) client_negotiation.cc:769] Sending connection context
0619 17:00:56.839394 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 17:00:56.839509 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17}
I20260619 17:00:56.840199 2293 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.231.188" port: 36683 } has no permanent_uuid. Determining permanent_uuid...
I20260619 17:00:56.840585 2297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.840482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:36683 (local address 127.0.0.1:40582)
0619 17:00:56.840536 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:56.840566 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.188:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 17:00:56.840901 2297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.840812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:36683 (local address 127.0.0.1:40586)
0619 17:00:56.840858 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:56.840881 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.188:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260619 17:00:56.841003 2293 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.231.188:36683: Network error: Client connection negotiation failed: client connection to 127.20.231.188:36683: connect: Connection refused (error 111)
I20260619 17:00:56.872154 2293 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.231.188" port: 36683 } attempt: 1
I20260619 17:00:56.872720 2297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.872562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:36683 (local address 127.0.0.1:40602)
0619 17:00:56.872632 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:56.872680 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.188:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260619 17:00:56.873178 2297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.873051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:36683 (local address 127.0.0.1:40612)
0619 17:00:56.873114 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:56.873148 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.188:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260619 17:00:56.873296 2293 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.231.188:36683: Network error: Client connection negotiation failed: client connection to 127.20.231.188:36683: connect: Connection refused (error 111)
W20260619 17:00:56.873837 2375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:00:56.874065 2375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:00:56.874112 2375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:00:56.875465 2375 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:00:56.875528 2375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:00:56.875555 2375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:00:56.875587 2375 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:00:56.875617 2375 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:00:56.877185 2375 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:46673
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:40615,127.20.231.189:36557,127.20.231.188:36683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.188.keytab
--principal=kudu/127.20.231.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.188:36683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.231.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:00:56.877480 2375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:00:56.877722 2375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:00:56.882759 2380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:00:56.882759 2383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:00:56.882755 2381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:00:56.882951 2375 server_base.cc:1034] running on GCE node
I20260619 17:00:56.883279 2375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:00:56.883566 2375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:00:56.884862 2375 hybrid_clock.cc:648] HybridClock initialized: now 1781888456884836 us; error 32 us; skew 500 ppm
Jun 19 17:00:56 dist-test-slave-vnbj krb5kdc[2212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888456, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:00:56.888155 2375 init.cc:377] Logged in from keytab as kudu/127.20.231.188@KRBTEST.COM (short username kudu)
I20260619 17:00:56.888649 2375 webserver.cc:460] Webserver started at http://127.20.231.188:41377/ using document root <none> and password file <none>
I20260619 17:00:56.888950 2375 fs_manager.cc:359] Metadata directory not provided
I20260619 17:00:56.889031 2375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:00:56.889194 2375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:00:56.890364 2375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-2/data/instance:
uuid: "20c6ed0646c444d085b3a85a04ca5b4d"
format_stamp: "Formatted at 2026-06-19 17:00:56 on dist-test-slave-vnbj"
server_key: "4de3c7eb923571ae61d2deb303aec26f"
server_key_iv: "0acb52d63d693328987c513b7990d630"
server_key_version: "encryptionkey@0"
I20260619 17:00:56.890734 2375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-2/wal/instance:
uuid: "20c6ed0646c444d085b3a85a04ca5b4d"
format_stamp: "Formatted at 2026-06-19 17:00:56 on dist-test-slave-vnbj"
server_key: "4de3c7eb923571ae61d2deb303aec26f"
server_key_iv: "0acb52d63d693328987c513b7990d630"
server_key_version: "encryptionkey@0"
I20260619 17:00:56.892230 2375 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 17:00:56.893023 2390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:00:56.893217 2375 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:00:56.893324 2375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-2/wal
uuid: "20c6ed0646c444d085b3a85a04ca5b4d"
format_stamp: "Formatted at 2026-06-19 17:00:56 on dist-test-slave-vnbj"
server_key: "4de3c7eb923571ae61d2deb303aec26f"
server_key_iv: "0acb52d63d693328987c513b7990d630"
server_key_version: "encryptionkey@0"
I20260619 17:00:56.893425 2375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:00:56.900450 2368 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.231.188" port: 36683 } attempt: 2
I20260619 17:00:56.901013 2370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.900857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:36683 (local address 127.0.0.1:40616)
0619 17:00:56.900927 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:56.900974 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.188:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 17:00:56.901360 2370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.901280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:36683 (local address 127.0.0.1:40626)
0619 17:00:56.901307 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:56.901342 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.188:36683: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
W20260619 17:00:56.901593 2368 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.231.188:36683: Network error: Client connection negotiation failed: client connection to 127.20.231.188:36683: connect: Connection refused (error 111)
I20260619 17:00:56.912839 2375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:00:56.915355 2375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:00:56.915530 2375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:00:56.919632 2375 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.188:36683
I20260619 17:00:56.919668 2442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.188:36683 every 8 connection(s)
I20260619 17:00:56.920044 2375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-2/data/info.pb
I20260619 17:00:56.920969 2443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:00:56.922601 2443 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.231.190" port: 40615 } has no permanent_uuid. Determining permanent_uuid...
I20260619 17:00:56.925323 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 2375
I20260619 17:00:56.925478 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781888301345144-21406-0/minicluster-data/master-2/wal/instance
I20260619 17:00:56.925653 21406 external_mini_cluster.cc:1428] Setting key 67c9edc1b81f5b844bf8f4992984e845
Jun 19 17:00:56 dist-test-slave-vnbj krb5kdc[2212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888456, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.188@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
Jun 19 17:00:56 dist-test-slave-vnbj krb5kdc[2212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.188@KRBTEST.COM
I20260619 17:00:56.932708 2296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.923437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37102 (local address 127.20.231.190:40615)
0619 17:00:56.923501 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:56.923503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:56.923850 (+ 347us) server_negotiation.cc:407] Connection header received
0619 17:00:56.924354 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:56.924355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:56.924368 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:56.924418 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:00:56.925061 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.925630 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.926964 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.927172 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.927261 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:56.929467 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:00:56.929469 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:00:56.929470 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:00:56.929478 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:00:56.931028 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:56.931460 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:56.931461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:56.931462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:56.931521 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:56.931883 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:56.931884 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:56.931884 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:56.932029 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:00:56.932098 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:56.932533 (+ 435us) server_negotiation.cc:299] Negotiation successful
0619 17:00:56.932625 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 17:00:56.933023 2445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.923408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:40615 (local address 127.0.0.1:37102)
0619 17:00:56.923612 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:56.923650 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 17:00:56.924241 (+ 591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:00:56.924498 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:00:56.924501 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:00:56.924757 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:00:56.924962 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:56.924966 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.925727 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.925728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:56.926815 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:56.926817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.927269 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.927270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:56.927421 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:56.927996 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:00:56.928004 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:00:56.929358 (+ 1354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:00:56.931128 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:00:56.931134 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:00:56.931136 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:00:56.931383 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:00:56.931590 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:00:56.931591 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:00:56.931591 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:00:56.931742 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:00:56.932135 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:00:56.932142 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:00:56.932374 (+ 232us) client_negotiation.cc:769] Sending connection context
0619 17:00:56.932489 (+ 115us) client_negotiation.cc:240] Negotiation successful
0619 17:00:56.932691 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":107,"threads_started":1}
I20260619 17:00:56.933677 2443 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.231.189" port: 36557 } has no permanent_uuid. Determining permanent_uuid...
I20260619 17:00:56.935261 2447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.926172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40628 (local address 127.20.231.188:36683)
0619 17:00:56.926343 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:56.926348 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:56.926374 (+ 26us) server_negotiation.cc:407] Connection header received
0619 17:00:56.926446 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:56.926449 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:56.926469 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:56.926549 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:00:56.927277 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.928188 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.928943 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.929372 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.929497 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:56.932140 (+ 2643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:00:56.932145 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:00:56.932145 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:00:56.932154 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:00:56.934083 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:56.934423 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:56.934426 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:56.934427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:56.934487 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:56.934693 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:56.934694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:56.934694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:56.934833 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:00:56.935022 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:56.935079 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 17:00:56.935150 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":98,"threads_started":1}
Jun 19 17:00:56 dist-test-slave-vnbj krb5kdc[2212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888456, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.188@KRBTEST.COM for kudu/127.20.231.189@KRBTEST.COM
Jun 19 17:00:56 dist-test-slave-vnbj krb5kdc[2212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:00:56.941093 2372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.934114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50366 (local address 127.20.231.189:36557)
0619 17:00:56.934155 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:56.934157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:56.934165 (+ 8us) server_negotiation.cc:407] Connection header received
0619 17:00:56.934192 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:56.934192 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:56.934202 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:56.934252 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:00:56.934552 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.935140 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.936000 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.936297 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.936373 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:56.938068 (+ 1695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:00:56.938069 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:00:56.938070 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:00:56.938077 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:00:56.940018 (+ 1941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:56.940345 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:56.940347 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:56.940347 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:56.940405 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:56.940614 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:56.940614 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:56.940614 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:56.940710 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:00:56.940757 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:56.940919 (+ 162us) server_negotiation.cc:299] Negotiation successful
0619 17:00:56.940978 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 17:00:56.941105 2445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.934041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.189:36557 (local address 127.0.0.1:50366)
0619 17:00:56.934089 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:56.934100 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 17:00:56.934137 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:00:56.934253 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:00:56.934253 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:00:56.934427 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:00:56.934461 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:56.934462 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.935235 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.935236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:56.935920 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:56.935922 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.936367 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.936368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:56.936431 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:56.936832 (+ 401us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:00:56.936836 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:00:56.937987 (+ 1151us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:00:56.940105 (+ 2118us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:00:56.940106 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:00:56.940106 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:00:56.940268 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:00:56.940484 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:00:56.940485 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:00:56.940485 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:00:56.940550 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:00:56.940774 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:00:56.940775 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:00:56.940822 (+ 47us) client_negotiation.cc:769] Sending connection context
0619 17:00:56.940902 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 17:00:56.940994 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 17:00:56.941654 2443 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.231.188" port: 36683 } has no permanent_uuid. Determining permanent_uuid...
I20260619 17:00:56.944623 2296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.936079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37112 (local address 127.20.231.190:40615)
0619 17:00:56.936115 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:56.936116 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:56.936149 (+ 33us) server_negotiation.cc:407] Connection header received
0619 17:00:56.936215 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:56.936215 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:56.936223 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:56.936266 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:00:56.936762 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.937255 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.938424 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.938697 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.938749 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:56.941477 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:00:56.941479 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:00:56.941479 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:00:56.941487 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:00:56.943261 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:56.943680 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:56.943681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:56.943681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:56.943741 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:56.943983 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:56.943983 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:56.943984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:56.944085 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:00:56.944230 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:56.944425 (+ 195us) server_negotiation.cc:299] Negotiation successful
0619 17:00:56.944503 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
Jun 19 17:00:56 dist-test-slave-vnbj krb5kdc[2212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888456, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.188@KRBTEST.COM for kudu/127.20.231.188@KRBTEST.COM
I20260619 17:00:56.948439 2293 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.231.188" port: 36683 } attempt: 2
I20260619 17:00:56.950132 2445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.941950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:36683 (local address 127.0.0.1:40642)
0619 17:00:56.941995 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:56.942006 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 17:00:56.942040 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:00:56.942150 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:00:56.942151 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:00:56.942295 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:00:56.942335 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:56.942337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.943157 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.943158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:56.943786 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:56.943789 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.944413 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.944414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:56.944466 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:56.944901 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:00:56.944906 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:00:56.946374 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:00:56.948137 (+ 1763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:00:56.948138 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:00:56.948139 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:00:56.948353 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:00:56.949299 (+ 946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:00:56.949300 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:00:56.949300 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:00:56.949368 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:00:56.949692 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:00:56.949693 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:00:56.949810 (+ 117us) client_negotiation.cc:769] Sending connection context
0619 17:00:56.949893 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 17:00:56.950004 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 17:00:56.950147 2447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.941980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40642 (local address 127.20.231.188:36683)
0619 17:00:56.942021 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:56.942023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:56.942031 (+ 8us) server_negotiation.cc:407] Connection header received
0619 17:00:56.942092 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:56.942093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:56.942102 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:56.942145 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:00:56.942418 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.943082 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.943881 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.944232 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.944309 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:56.946454 (+ 2145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:00:56.946455 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:00:56.946455 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:00:56.946461 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:00:56.948021 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:56.949111 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:56.949112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:56.949112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:56.949203 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:56.949449 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:56.949450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:56.949450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:56.949599 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:00:56.949682 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:56.949918 (+ 236us) server_negotiation.cc:299] Negotiation successful
0619 17:00:56.950051 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":69,"server-negotiator.queue_time_us":23}
Jun 19 17:00:56 dist-test-slave-vnbj krb5kdc[2212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.189@KRBTEST.COM
I20260619 17:00:56.953012 2443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20c6ed0646c444d085b3a85a04ca5b4d: Bootstrap starting.
Jun 19 17:00:56 dist-test-slave-vnbj krb5kdc[2212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888456, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for kudu/127.20.231.188@KRBTEST.COM
I20260619 17:00:56.953837 2443 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20c6ed0646c444d085b3a85a04ca5b4d: Neither blocks nor log segments found. Creating new log.
I20260619 17:00:56.954226 2443 log.cc:824] T 00000000000000000000000000000000 P 20c6ed0646c444d085b3a85a04ca5b4d: Log is configured to *not* fsync() on all Append() calls
I20260619 17:00:56.954784 2372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.945700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50376 (local address 127.20.231.189:36557)
0619 17:00:56.945741 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:56.945743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:56.945775 (+ 32us) server_negotiation.cc:407] Connection header received
0619 17:00:56.945800 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:56.945801 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:56.945809 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:56.945851 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:00:56.946378 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.947137 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.948143 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.948475 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.948586 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:56.951259 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:00:56.951260 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:00:56.951260 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:00:56.951268 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:00:56.953096 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:56.953473 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:56.953474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:56.953475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:56.953531 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:56.953769 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:56.953770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:56.953770 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:56.953924 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:00:56.954142 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:56.954569 (+ 427us) server_negotiation.cc:299] Negotiation successful
0619 17:00:56.954651 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 17:00:56.955256 2443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20c6ed0646c444d085b3a85a04ca5b4d: No bootstrap required, opened a new log
I20260619 17:00:56.956422 21406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 17:00:56.957317 2297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.948694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:36683 (local address 127.0.0.1:40644)
0619 17:00:56.948767 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:56.948779 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 17:00:56.948818 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:00:56.949223 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:00:56.949224 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:00:56.949385 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:00:56.949471 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:56.949474 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.950914 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.950915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:56.951578 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:56.951581 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.952051 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.952052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:56.952119 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:56.952521 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:00:56.952526 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:00:56.953978 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:00:56.956140 (+ 2162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:00:56.956141 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:00:56.956141 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:00:56.956374 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:00:56.956602 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:00:56.956603 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:00:56.956603 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:00:56.956669 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:00:56.956955 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:00:56.956956 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:00:56.957006 (+ 50us) client_negotiation.cc:769] Sending connection context
0619 17:00:56.957115 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 17:00:56.957221 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260619 17:00:56.957360 2448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.948896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40644 (local address 127.20.231.188:36683)
0619 17:00:56.949080 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:56.949082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:56.949090 (+ 8us) server_negotiation.cc:407] Connection header received
0619 17:00:56.949110 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:56.949111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:56.949122 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:56.949203 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:00:56.949568 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.950772 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.951666 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.951965 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.952036 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:56.954076 (+ 2040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:00:56.954077 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:00:56.954077 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:00:56.954085 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:00:56.956047 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:56.956459 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:56.956460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:56.956460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:56.956541 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:56.956743 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:56.956743 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:56.956744 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:56.956882 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:00:56.956970 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:56.957146 (+ 176us) server_negotiation.cc:299] Negotiation successful
0619 17:00:56.957222 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":106,"threads_started":1}
I20260619 17:00:56.957548 2443 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20c6ed0646c444d085b3a85a04ca5b4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aaa4258be0740ed956f4dc48f232e4d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 40615 } } peers { permanent_uuid: "854fd43d6785425fac06d05c4fd6923e" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 36557 } } peers { permanent_uuid: "20c6ed0646c444d085b3a85a04ca5b4d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 36683 } }
I20260619 17:00:56.957723 2443 raft_consensus.cc:374] T 00000000000000000000000000000000 P 20c6ed0646c444d085b3a85a04ca5b4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:00:56.957762 2443 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20c6ed0646c444d085b3a85a04ca5b4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20c6ed0646c444d085b3a85a04ca5b4d, State: Initialized, Role: FOLLOWER
I20260619 17:00:56.957898 2443 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20c6ed0646c444d085b3a85a04ca5b4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aaa4258be0740ed956f4dc48f232e4d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 40615 } } peers { permanent_uuid: "854fd43d6785425fac06d05c4fd6923e" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 36557 } } peers { permanent_uuid: "20c6ed0646c444d085b3a85a04ca5b4d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 36683 } }
I20260619 17:00:56.958235 2443 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20c6ed0646c444d085b3a85a04ca5b4d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:00:56.958617 2454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20c6ed0646c444d085b3a85a04ca5b4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aaa4258be0740ed956f4dc48f232e4d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 40615 } } peers { permanent_uuid: "854fd43d6785425fac06d05c4fd6923e" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 36557 } } peers { permanent_uuid: "20c6ed0646c444d085b3a85a04ca5b4d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 36683 } } }
I20260619 17:00:56.958716 2454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20c6ed0646c444d085b3a85a04ca5b4d [sys.catalog]: This master's current role is: FOLLOWER
I20260619 17:00:56.959034 2293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3aaa4258be0740ed956f4dc48f232e4d: Bootstrap starting.
I20260619 17:00:56.959860 2293 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3aaa4258be0740ed956f4dc48f232e4d: Neither blocks nor log segments found. Creating new log.
I20260619 17:00:56.960172 2293 log.cc:824] T 00000000000000000000000000000000 P 3aaa4258be0740ed956f4dc48f232e4d: Log is configured to *not* fsync() on all Append() calls
W20260619 17:00:56.960176 2467 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 20c6ed0646c444d085b3a85a04ca5b4d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 17:00:56.960639 2467 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 17:00:56.962005 2293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3aaa4258be0740ed956f4dc48f232e4d: No bootstrap required, opened a new log
I20260619 17:00:56.961158 2375 master_runner.cc:186] Error getting master registration for 127.20.231.190:40615: OK, instance_id { permanent_uuid: "3aaa4258be0740ed956f4dc48f232e4d" instance_seqno: 1781888456604263 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 17:00:56.963624 2375 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 17:00:56.964118 2293 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3aaa4258be0740ed956f4dc48f232e4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aaa4258be0740ed956f4dc48f232e4d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 40615 } } peers { permanent_uuid: "854fd43d6785425fac06d05c4fd6923e" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 36557 } } peers { permanent_uuid: "20c6ed0646c444d085b3a85a04ca5b4d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 36683 } }
I20260619 17:00:56.964272 2293 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3aaa4258be0740ed956f4dc48f232e4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:00:56.964315 2293 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3aaa4258be0740ed956f4dc48f232e4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3aaa4258be0740ed956f4dc48f232e4d, State: Initialized, Role: FOLLOWER
I20260619 17:00:56.964433 2293 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3aaa4258be0740ed956f4dc48f232e4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aaa4258be0740ed956f4dc48f232e4d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 40615 } } peers { permanent_uuid: "854fd43d6785425fac06d05c4fd6923e" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 36557 } } peers { permanent_uuid: "20c6ed0646c444d085b3a85a04ca5b4d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 36683 } }
I20260619 17:00:56.964775 2293 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3aaa4258be0740ed956f4dc48f232e4d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:00:56.967490 2225 master_runner.cc:186] Error getting master registration for 127.20.231.189:36557: OK, instance_id { permanent_uuid: "854fd43d6785425fac06d05c4fd6923e" instance_seqno: 1781888456753632 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 17:00:56.971159 2225 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 17:00:56.971302 2296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.960437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37118 (local address 127.20.231.190:40615)
0619 17:00:56.960480 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:56.960482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:56.960940 (+ 458us) server_negotiation.cc:407] Connection header received
0619 17:00:56.961051 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:56.961052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:56.961062 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:56.961092 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:00:56.961817 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.962495 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.964997 (+ 2502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.965292 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.965335 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:56.967903 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:00:56.967905 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:00:56.967905 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:00:56.967912 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:00:56.969653 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:56.970113 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:56.970115 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:56.970115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:56.970185 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:56.970524 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:56.970526 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:56.970526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:56.970673 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:00:56.970755 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:56.971033 (+ 278us) server_negotiation.cc:299] Negotiation successful
0619 17:00:56.971151 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 17:00:56.972491 2482 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3aaa4258be0740ed956f4dc48f232e4d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 17:00:56.972551 2482 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 17:00:56.972757 2471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3aaa4258be0740ed956f4dc48f232e4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aaa4258be0740ed956f4dc48f232e4d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 40615 } } peers { permanent_uuid: "854fd43d6785425fac06d05c4fd6923e" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 36557 } } peers { permanent_uuid: "20c6ed0646c444d085b3a85a04ca5b4d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 36683 } } }
I20260619 17:00:56.972836 2471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3aaa4258be0740ed956f4dc48f232e4d [sys.catalog]: This master's current role is: FOLLOWER
I20260619 17:00:56.973031 2372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.957926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50380 (local address 127.20.231.189:36557)
0619 17:00:56.957963 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:56.957965 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:56.958494 (+ 529us) server_negotiation.cc:407] Connection header received
0619 17:00:56.958543 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:56.958545 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:56.958556 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:56.958604 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:00:56.959615 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.960367 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.965798 (+ 5431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.966131 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.966196 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:56.968656 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:00:56.968657 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:00:56.968658 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:00:56.968666 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:00:56.970276 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:56.972111 (+ 1835us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:56.972112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:56.972113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:56.972176 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:56.972404 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:56.972405 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:56.972405 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:56.972540 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:00:56.972630 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:56.972833 (+ 203us) server_negotiation.cc:299] Negotiation successful
0619 17:00:56.972931 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 17:00:56.974319 2448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.960498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40658 (local address 127.20.231.188:36683)
0619 17:00:56.960578 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:56.960580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:56.960703 (+ 123us) server_negotiation.cc:407] Connection header received
0619 17:00:56.961082 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:56.961083 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:56.961095 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:56.961123 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:00:56.963773 (+ 2650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.964526 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.965730 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.966074 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.966139 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:56.968953 (+ 2814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:00:56.968954 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:00:56.968954 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:00:56.968962 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:00:56.971547 (+ 2585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:56.973389 (+ 1842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:56.973390 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:56.973390 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:56.973455 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:56.973705 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:56.973706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:56.973707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:56.973867 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:00:56.973974 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:56.974132 (+ 158us) server_negotiation.cc:299] Negotiation successful
0619 17:00:56.974227 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 17:00:56.992762 2368 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.231.188" port: 36683 } attempt: 3
Jun 19 17:00:56 dist-test-slave-vnbj krb5kdc[2212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888456, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.189@KRBTEST.COM for kudu/127.20.231.188@KRBTEST.COM
I20260619 17:00:57.002012 2370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.993125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:36683 (local address 127.0.0.1:40668)
0619 17:00:56.993179 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 17:00:56.993191 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 17:00:56.993237 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:00:56.993448 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:00:56.993449 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:00:56.993649 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:00:56.993703 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:56.993705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.994993 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.994994 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:56.995796 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:00:56.995800 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.996324 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.996325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:00:56.996411 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:56.996952 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:00:56.996958 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:00:56.998601 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:00:57.000777 (+ 2176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:00:57.000778 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:00:57.000779 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:00:57.001023 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:00:57.001249 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:00:57.001250 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:00:57.001250 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:00:57.001320 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:00:57.001657 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:00:57.001658 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:00:57.001723 (+ 65us) client_negotiation.cc:769] Sending connection context
0619 17:00:57.001777 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 17:00:57.001888 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 17:00:57.002691 2448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:56.993217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40668 (local address 127.20.231.188:36683)
0619 17:00:56.993276 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:56.993278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:56.993289 (+ 11us) server_negotiation.cc:407] Connection header received
0619 17:00:56.993320 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:56.993321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:56.993332 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:56.993423 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:00:56.994074 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.994893 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.995907 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:56.996223 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:56.996287 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:56.998825 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:00:56.998827 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:00:56.998827 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:00:56.998836 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:00:57.000673 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:57.001098 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:57.001100 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:57.001100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:57.001169 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:57.001390 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:57.001390 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:57.001391 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:57.001554 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:00:57.002444 (+ 890us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:57.002492 (+ 48us) server_negotiation.cc:299] Negotiation successful
0619 17:00:57.002580 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 17:00:57.004570 2368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 854fd43d6785425fac06d05c4fd6923e: Bootstrap starting.
I20260619 17:00:57.005327 2368 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 854fd43d6785425fac06d05c4fd6923e: Neither blocks nor log segments found. Creating new log.
I20260619 17:00:57.005649 2368 log.cc:824] T 00000000000000000000000000000000 P 854fd43d6785425fac06d05c4fd6923e: Log is configured to *not* fsync() on all Append() calls
I20260619 17:00:57.006560 2368 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 854fd43d6785425fac06d05c4fd6923e: No bootstrap required, opened a new log
I20260619 17:00:57.008631 2368 raft_consensus.cc:348] T 00000000000000000000000000000000 P 854fd43d6785425fac06d05c4fd6923e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aaa4258be0740ed956f4dc48f232e4d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 40615 } } peers { permanent_uuid: "854fd43d6785425fac06d05c4fd6923e" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 36557 } } peers { permanent_uuid: "20c6ed0646c444d085b3a85a04ca5b4d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 36683 } }
I20260619 17:00:57.008778 2368 raft_consensus.cc:374] T 00000000000000000000000000000000 P 854fd43d6785425fac06d05c4fd6923e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:00:57.008811 2368 raft_consensus.cc:729] T 00000000000000000000000000000000 P 854fd43d6785425fac06d05c4fd6923e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 854fd43d6785425fac06d05c4fd6923e, State: Initialized, Role: FOLLOWER
I20260619 17:00:57.008971 2368 consensus_queue.cc:260] T 00000000000000000000000000000000 P 854fd43d6785425fac06d05c4fd6923e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aaa4258be0740ed956f4dc48f232e4d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 40615 } } peers { permanent_uuid: "854fd43d6785425fac06d05c4fd6923e" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 36557 } } peers { permanent_uuid: "20c6ed0646c444d085b3a85a04ca5b4d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 36683 } }
I20260619 17:00:57.009297 2368 sys_catalog.cc:564] T 00000000000000000000000000000000 P 854fd43d6785425fac06d05c4fd6923e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:00:57.009266 2485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 854fd43d6785425fac06d05c4fd6923e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aaa4258be0740ed956f4dc48f232e4d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 40615 } } peers { permanent_uuid: "854fd43d6785425fac06d05c4fd6923e" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 36557 } } peers { permanent_uuid: "20c6ed0646c444d085b3a85a04ca5b4d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 36683 } } }
I20260619 17:00:57.009483 2485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 854fd43d6785425fac06d05c4fd6923e [sys.catalog]: This master's current role is: FOLLOWER
W20260619 17:00:57.011438 2496 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 854fd43d6785425fac06d05c4fd6923e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 17:00:57.011559 2496 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 17:00:57.013857 2300 master_runner.cc:263] No leader master found from master 854fd43d6785425fac06d05c4fd6923e
I20260619 17:00:57.013931 2300 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260619 17:00:57.102197 2246 tablet.cc:2369] T 00000000000000000000000000000000 P 3aaa4258be0740ed956f4dc48f232e4d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 17:00:57.166275 2396 tablet.cc:2369] T 00000000000000000000000000000000 P 20c6ed0646c444d085b3a85a04ca5b4d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 17:00:57.194774 2485 raft_consensus.cc:482] T 00000000000000000000000000000000 P 854fd43d6785425fac06d05c4fd6923e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 17:00:57.194958 2485 raft_consensus.cc:504] T 00000000000000000000000000000000 P 854fd43d6785425fac06d05c4fd6923e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aaa4258be0740ed956f4dc48f232e4d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 40615 } } peers { permanent_uuid: "854fd43d6785425fac06d05c4fd6923e" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 36557 } } peers { permanent_uuid: "20c6ed0646c444d085b3a85a04ca5b4d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 36683 } }
I20260619 17:00:57.195287 2485 leader_election.cc:288] T 00000000000000000000000000000000 P 854fd43d6785425fac06d05c4fd6923e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3aaa4258be0740ed956f4dc48f232e4d (127.20.231.190:40615), 20c6ed0646c444d085b3a85a04ca5b4d (127.20.231.188:36683)
I20260619 17:00:57.195775 2417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "854fd43d6785425fac06d05c4fd6923e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c6ed0646c444d085b3a85a04ca5b4d" is_pre_election: true
I20260619 17:00:57.195763 2267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "854fd43d6785425fac06d05c4fd6923e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aaa4258be0740ed956f4dc48f232e4d" is_pre_election: true
I20260619 17:00:57.195986 2417 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 20c6ed0646c444d085b3a85a04ca5b4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 854fd43d6785425fac06d05c4fd6923e in term 0.
I20260619 17:00:57.196013 2267 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3aaa4258be0740ed956f4dc48f232e4d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 854fd43d6785425fac06d05c4fd6923e in term 0.
I20260619 17:00:57.196264 2316 leader_election.cc:302] T 00000000000000000000000000000000 P 854fd43d6785425fac06d05c4fd6923e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aaa4258be0740ed956f4dc48f232e4d, 854fd43d6785425fac06d05c4fd6923e; no voters:
I20260619 17:00:57.196429 2485 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 854fd43d6785425fac06d05c4fd6923e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 17:00:57.196492 2485 raft_consensus.cc:482] T 00000000000000000000000000000000 P 854fd43d6785425fac06d05c4fd6923e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 17:00:57.196523 2485 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 854fd43d6785425fac06d05c4fd6923e [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:00:57.197504 2485 raft_consensus.cc:504] T 00000000000000000000000000000000 P 854fd43d6785425fac06d05c4fd6923e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aaa4258be0740ed956f4dc48f232e4d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 40615 } } peers { permanent_uuid: "854fd43d6785425fac06d05c4fd6923e" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 36557 } } peers { permanent_uuid: "20c6ed0646c444d085b3a85a04ca5b4d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 36683 } }
I20260619 17:00:57.197669 2485 leader_election.cc:288] T 00000000000000000000000000000000 P 854fd43d6785425fac06d05c4fd6923e [CANDIDATE]: Term 1 election: Requested vote from peers 3aaa4258be0740ed956f4dc48f232e4d (127.20.231.190:40615), 20c6ed0646c444d085b3a85a04ca5b4d (127.20.231.188:36683)
I20260619 17:00:57.197835 2417 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "854fd43d6785425fac06d05c4fd6923e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20c6ed0646c444d085b3a85a04ca5b4d"
I20260619 17:00:57.197866 2267 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "854fd43d6785425fac06d05c4fd6923e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3aaa4258be0740ed956f4dc48f232e4d"
I20260619 17:00:57.197937 2267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3aaa4258be0740ed956f4dc48f232e4d [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:00:57.197934 2417 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20c6ed0646c444d085b3a85a04ca5b4d [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:00:57.198670 2267 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 3aaa4258be0740ed956f4dc48f232e4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 854fd43d6785425fac06d05c4fd6923e in term 1.
I20260619 17:00:57.198771 2417 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 20c6ed0646c444d085b3a85a04ca5b4d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 854fd43d6785425fac06d05c4fd6923e in term 1.
I20260619 17:00:57.198868 2316 leader_election.cc:302] T 00000000000000000000000000000000 P 854fd43d6785425fac06d05c4fd6923e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3aaa4258be0740ed956f4dc48f232e4d, 854fd43d6785425fac06d05c4fd6923e; no voters:
I20260619 17:00:57.198966 2485 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 854fd43d6785425fac06d05c4fd6923e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:00:57.199137 2485 raft_consensus.cc:686] T 00000000000000000000000000000000 P 854fd43d6785425fac06d05c4fd6923e [term 1 LEADER]: Becoming Leader. State: Replica: 854fd43d6785425fac06d05c4fd6923e, State: Running, Role: LEADER
I20260619 17:00:57.199252 2485 consensus_queue.cc:237] T 00000000000000000000000000000000 P 854fd43d6785425fac06d05c4fd6923e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aaa4258be0740ed956f4dc48f232e4d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 40615 } } peers { permanent_uuid: "854fd43d6785425fac06d05c4fd6923e" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 36557 } } peers { permanent_uuid: "20c6ed0646c444d085b3a85a04ca5b4d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 36683 } }
I20260619 17:00:57.199862 2498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 854fd43d6785425fac06d05c4fd6923e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 854fd43d6785425fac06d05c4fd6923e. Latest consensus state: current_term: 1 leader_uuid: "854fd43d6785425fac06d05c4fd6923e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aaa4258be0740ed956f4dc48f232e4d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 40615 } } peers { permanent_uuid: "854fd43d6785425fac06d05c4fd6923e" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 36557 } } peers { permanent_uuid: "20c6ed0646c444d085b3a85a04ca5b4d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 36683 } } }
I20260619 17:00:57.199980 2498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 854fd43d6785425fac06d05c4fd6923e [sys.catalog]: This master's current role is: LEADER
I20260619 17:00:57.200227 2500 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 17:00:57.200415 2500 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 17:00:57.201943 2267 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 3aaa4258be0740ed956f4dc48f232e4d [term 1 FOLLOWER]: Refusing update from remote peer 854fd43d6785425fac06d05c4fd6923e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 17:00:57.202025 2417 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 20c6ed0646c444d085b3a85a04ca5b4d [term 1 FOLLOWER]: Refusing update from remote peer 854fd43d6785425fac06d05c4fd6923e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 17:00:57.202239 2498 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 854fd43d6785425fac06d05c4fd6923e [LEADER]: Connected to new peer: Peer: permanent_uuid: "3aaa4258be0740ed956f4dc48f232e4d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 40615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 17:00:57.202351 2485 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 854fd43d6785425fac06d05c4fd6923e [LEADER]: Connected to new peer: Peer: permanent_uuid: "20c6ed0646c444d085b3a85a04ca5b4d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 36683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 17:00:57.203578 2471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3aaa4258be0740ed956f4dc48f232e4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 854fd43d6785425fac06d05c4fd6923e. Latest consensus state: current_term: 1 leader_uuid: "854fd43d6785425fac06d05c4fd6923e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aaa4258be0740ed956f4dc48f232e4d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 40615 } } peers { permanent_uuid: "854fd43d6785425fac06d05c4fd6923e" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 36557 } } peers { permanent_uuid: "20c6ed0646c444d085b3a85a04ca5b4d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 36683 } } }
I20260619 17:00:57.203689 2471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3aaa4258be0740ed956f4dc48f232e4d [sys.catalog]: This master's current role is: FOLLOWER
I20260619 17:00:57.203972 2454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20c6ed0646c444d085b3a85a04ca5b4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 854fd43d6785425fac06d05c4fd6923e. Latest consensus state: current_term: 1 leader_uuid: "854fd43d6785425fac06d05c4fd6923e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aaa4258be0740ed956f4dc48f232e4d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 40615 } } peers { permanent_uuid: "854fd43d6785425fac06d05c4fd6923e" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 36557 } } peers { permanent_uuid: "20c6ed0646c444d085b3a85a04ca5b4d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 36683 } } }
I20260619 17:00:57.204042 2454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20c6ed0646c444d085b3a85a04ca5b4d [sys.catalog]: This master's current role is: FOLLOWER
I20260619 17:00:57.204566 2498 sys_catalog.cc:455] T 00000000000000000000000000000000 P 854fd43d6785425fac06d05c4fd6923e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "854fd43d6785425fac06d05c4fd6923e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aaa4258be0740ed956f4dc48f232e4d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 40615 } } peers { permanent_uuid: "854fd43d6785425fac06d05c4fd6923e" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 36557 } } peers { permanent_uuid: "20c6ed0646c444d085b3a85a04ca5b4d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 36683 } } }
I20260619 17:00:57.204667 2498 sys_catalog.cc:458] T 00000000000000000000000000000000 P 854fd43d6785425fac06d05c4fd6923e [sys.catalog]: This master's current role is: LEADER
I20260619 17:00:57.204751 2485 sys_catalog.cc:455] T 00000000000000000000000000000000 P 854fd43d6785425fac06d05c4fd6923e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "854fd43d6785425fac06d05c4fd6923e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aaa4258be0740ed956f4dc48f232e4d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 40615 } } peers { permanent_uuid: "854fd43d6785425fac06d05c4fd6923e" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 36557 } } peers { permanent_uuid: "20c6ed0646c444d085b3a85a04ca5b4d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 36683 } } }
I20260619 17:00:57.204807 2485 sys_catalog.cc:458] T 00000000000000000000000000000000 P 854fd43d6785425fac06d05c4fd6923e [sys.catalog]: This master's current role is: LEADER
I20260619 17:00:57.204905 2454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20c6ed0646c444d085b3a85a04ca5b4d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "854fd43d6785425fac06d05c4fd6923e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aaa4258be0740ed956f4dc48f232e4d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 40615 } } peers { permanent_uuid: "854fd43d6785425fac06d05c4fd6923e" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 36557 } } peers { permanent_uuid: "20c6ed0646c444d085b3a85a04ca5b4d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 36683 } } }
I20260619 17:00:57.204967 2454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20c6ed0646c444d085b3a85a04ca5b4d [sys.catalog]: This master's current role is: FOLLOWER
I20260619 17:00:57.205062 2471 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3aaa4258be0740ed956f4dc48f232e4d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "854fd43d6785425fac06d05c4fd6923e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3aaa4258be0740ed956f4dc48f232e4d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 40615 } } peers { permanent_uuid: "854fd43d6785425fac06d05c4fd6923e" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 36557 } } peers { permanent_uuid: "20c6ed0646c444d085b3a85a04ca5b4d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 36683 } } }
I20260619 17:00:57.205066 2503 mvcc.cc:205] Tried to move back new op lower bound from 7298615120696406016 to 7298615120688812032. Current Snapshot: MvccSnapshot[applied={T|T < 7298615120696406016}]
I20260619 17:00:57.205127 2471 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3aaa4258be0740ed956f4dc48f232e4d [sys.catalog]: This master's current role is: FOLLOWER
I20260619 17:00:57.205308 2502 mvcc.cc:205] Tried to move back new op lower bound from 7298615120696406016 to 7298615120688812032. Current Snapshot: MvccSnapshot[applied={T|T < 7298615120696406016}]
I20260619 17:00:57.205775 2500 catalog_manager.cc:1347] Generated new cluster ID: bf771e8f2a3f46c88b4205f04b615e73
I20260619 17:00:57.205827 2500 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 17:00:57.214226 2500 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 17:00:57.215078 2500 catalog_manager.cc:1504] Loading token signing keys...
I20260619 17:00:57.219029 2500 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 854fd43d6785425fac06d05c4fd6923e: Generated new TSK 0
I20260619 17:00:57.219261 2500 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 17:00:57.961140 2467 catalog_manager.cc:1259] Loaded cluster ID: bf771e8f2a3f46c88b4205f04b615e73
I20260619 17:00:57.961285 2467 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 20c6ed0646c444d085b3a85a04ca5b4d: loading cluster ID for follower catalog manager: success
I20260619 17:00:57.962141 2467 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 20c6ed0646c444d085b3a85a04ca5b4d: acquiring CA information for follower catalog manager: success
I20260619 17:00:57.962463 2467 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 20c6ed0646c444d085b3a85a04ca5b4d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 17:00:57.973090 2482 catalog_manager.cc:1259] Loaded cluster ID: bf771e8f2a3f46c88b4205f04b615e73
I20260619 17:00:57.973165 2482 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 3aaa4258be0740ed956f4dc48f232e4d: loading cluster ID for follower catalog manager: success
I20260619 17:00:57.974112 2482 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 3aaa4258be0740ed956f4dc48f232e4d: acquiring CA information for follower catalog manager: success
I20260619 17:00:57.974453 2482 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 3aaa4258be0740ed956f4dc48f232e4d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 17:00:58.288115 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 2225
I20260619 17:00:58.293972 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 2300
I20260619 17:00:58.299472 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 2375
2026-06-19T17:00:58Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3050 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 17:00:58 dist-test-slave-vnbj krb5kdc[2515](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 17:00:58 dist-test-slave-vnbj krb5kdc[2515](info): set up 2 sockets
Jun 19 17:00:58 dist-test-slave-vnbj krb5kdc[2515](info): commencing operation
krb5kdc: starting...
W20260619 17:00:59.422996 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 17:00:59 dist-test-slave-vnbj krb5kdc[2515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T17:00:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T17:00:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.231.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
I20260619 17:00:59.551216 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:35215
--webserver_interface=127.20.231.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:44853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:35215
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:00:59.636112 2528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:00:59.636415 2528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:00:59.636452 2528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:00:59.638080 2528 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:00:59.638134 2528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:00:59.638154 2528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:00:59.638167 2528 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:00:59.638206 2528 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:00:59.640566 2528 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:44853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:35215
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:35215
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:00:59.640872 2528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:00:59.641122 2528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:00:59.646159 2533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:00:59.646162 2536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:00:59.646159 2534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:00:59.646456 2528 server_base.cc:1034] running on GCE node
I20260619 17:00:59.646723 2528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:00:59.647049 2528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:00:59.648245 2528 hybrid_clock.cc:648] HybridClock initialized: now 1781888459648203 us; error 42 us; skew 500 ppm
Jun 19 17:00:59 dist-test-slave-vnbj krb5kdc[2515](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888459, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:00:59.651021 2528 init.cc:377] Logged in from keytab as kudu/127.20.231.190@KRBTEST.COM (short username kudu)
I20260619 17:00:59.651538 2528 webserver.cc:460] Webserver started at http://127.20.231.190:36335/ using document root <none> and password file <none>
I20260619 17:00:59.651778 2528 fs_manager.cc:359] Metadata directory not provided
I20260619 17:00:59.651849 2528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:00:59.651989 2528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:00:59.653142 2528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "6d4b2d8b65bf4eb8a6ea9fc70ca5e122"
format_stamp: "Formatted at 2026-06-19 17:00:59 on dist-test-slave-vnbj"
server_key: "65b694fc808fd3db133cc1321769f30f"
server_key_iv: "b807506c74e017417a86f5b8387e1001"
server_key_version: "encryptionkey@0"
I20260619 17:00:59.653545 2528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "6d4b2d8b65bf4eb8a6ea9fc70ca5e122"
format_stamp: "Formatted at 2026-06-19 17:00:59 on dist-test-slave-vnbj"
server_key: "65b694fc808fd3db133cc1321769f30f"
server_key_iv: "b807506c74e017417a86f5b8387e1001"
server_key_version: "encryptionkey@0"
I20260619 17:00:59.655107 2528 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 17:00:59.655920 2543 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:00:59.656098 2528 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 17:00:59.656216 2528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "6d4b2d8b65bf4eb8a6ea9fc70ca5e122"
format_stamp: "Formatted at 2026-06-19 17:00:59 on dist-test-slave-vnbj"
server_key: "65b694fc808fd3db133cc1321769f30f"
server_key_iv: "b807506c74e017417a86f5b8387e1001"
server_key_version: "encryptionkey@0"
I20260619 17:00:59.656323 2528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:00:59.664397 2528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:00:59.666347 2528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:00:59.666525 2528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:00:59.670696 2528 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:35215
I20260619 17:00:59.670728 2595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:35215 every 8 connection(s)
I20260619 17:00:59.671149 2528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 17:00:59.672044 2596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:00:59.674340 2596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d4b2d8b65bf4eb8a6ea9fc70ca5e122: Bootstrap starting.
I20260619 17:00:59.675002 2596 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d4b2d8b65bf4eb8a6ea9fc70ca5e122: Neither blocks nor log segments found. Creating new log.
I20260619 17:00:59.675292 2596 log.cc:824] T 00000000000000000000000000000000 P 6d4b2d8b65bf4eb8a6ea9fc70ca5e122: Log is configured to *not* fsync() on all Append() calls
I20260619 17:00:59.676011 2596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d4b2d8b65bf4eb8a6ea9fc70ca5e122: No bootstrap required, opened a new log
I20260619 17:00:59.676462 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 2528
I20260619 17:00:59.676555 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 17:00:59.676733 21406 external_mini_cluster.cc:1428] Setting key 4f9cbed6aaa5f9f13916eb183d43d925
I20260619 17:00:59.678162 2596 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d4b2d8b65bf4eb8a6ea9fc70ca5e122 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d4b2d8b65bf4eb8a6ea9fc70ca5e122" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 35215 } }
I20260619 17:00:59.678295 2596 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d4b2d8b65bf4eb8a6ea9fc70ca5e122 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:00:59.678318 2596 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d4b2d8b65bf4eb8a6ea9fc70ca5e122 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d4b2d8b65bf4eb8a6ea9fc70ca5e122, State: Initialized, Role: FOLLOWER
I20260619 17:00:59.678454 2596 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d4b2d8b65bf4eb8a6ea9fc70ca5e122 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d4b2d8b65bf4eb8a6ea9fc70ca5e122" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 35215 } }
I20260619 17:00:59.678552 2596 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d4b2d8b65bf4eb8a6ea9fc70ca5e122 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 17:00:59.678596 2596 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d4b2d8b65bf4eb8a6ea9fc70ca5e122 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 17:00:59.678670 2596 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d4b2d8b65bf4eb8a6ea9fc70ca5e122 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:00:59.679602 2596 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d4b2d8b65bf4eb8a6ea9fc70ca5e122 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d4b2d8b65bf4eb8a6ea9fc70ca5e122" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 35215 } }
I20260619 17:00:59.679735 2596 leader_election.cc:302] T 00000000000000000000000000000000 P 6d4b2d8b65bf4eb8a6ea9fc70ca5e122 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d4b2d8b65bf4eb8a6ea9fc70ca5e122; no voters:
I20260619 17:00:59.679936 2596 leader_election.cc:288] T 00000000000000000000000000000000 P 6d4b2d8b65bf4eb8a6ea9fc70ca5e122 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 17:00:59.679966 2601 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d4b2d8b65bf4eb8a6ea9fc70ca5e122 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:00:59.680069 2601 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d4b2d8b65bf4eb8a6ea9fc70ca5e122 [term 1 LEADER]: Becoming Leader. State: Replica: 6d4b2d8b65bf4eb8a6ea9fc70ca5e122, State: Running, Role: LEADER
I20260619 17:00:59.680200 2596 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d4b2d8b65bf4eb8a6ea9fc70ca5e122 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:00:59.680241 2601 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d4b2d8b65bf4eb8a6ea9fc70ca5e122 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d4b2d8b65bf4eb8a6ea9fc70ca5e122" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 35215 } }
I20260619 17:00:59.680595 2601 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d4b2d8b65bf4eb8a6ea9fc70ca5e122 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d4b2d8b65bf4eb8a6ea9fc70ca5e122. Latest consensus state: current_term: 1 leader_uuid: "6d4b2d8b65bf4eb8a6ea9fc70ca5e122" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d4b2d8b65bf4eb8a6ea9fc70ca5e122" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 35215 } } }
I20260619 17:00:59.680608 2602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d4b2d8b65bf4eb8a6ea9fc70ca5e122 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d4b2d8b65bf4eb8a6ea9fc70ca5e122" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d4b2d8b65bf4eb8a6ea9fc70ca5e122" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 35215 } } }
I20260619 17:00:59.680728 2601 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d4b2d8b65bf4eb8a6ea9fc70ca5e122 [sys.catalog]: This master's current role is: LEADER
I20260619 17:00:59.680779 2602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d4b2d8b65bf4eb8a6ea9fc70ca5e122 [sys.catalog]: This master's current role is: LEADER
I20260619 17:00:59.681528 2615 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 17:00:59.682109 2615 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 17:00:59.683899 2615 catalog_manager.cc:1347] Generated new cluster ID: d8ea42cb4e43402c8c510c08bef2aecb
I20260619 17:00:59.683957 2615 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 17:00:59 dist-test-slave-vnbj krb5kdc[2515](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:00:59.690002 2600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:00:59.677200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54194 (local address 127.20.231.190:35215)
0619 17:00:59.677383 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 17:00:59.677391 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 17:00:59.677419 (+ 28us) server_negotiation.cc:407] Connection header received
0619 17:00:59.678151 (+ 732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:00:59.678160 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:00:59.678244 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:00:59.678438 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:00:59.679391 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:59.680702 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:59.681954 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:00:59.682755 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:00:59.682941 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:00:59.685983 (+ 3042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:00:59.685988 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:00:59.685988 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:00:59.686002 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:00:59.688158 (+ 2156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:59.688648 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:59.688652 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:59.688652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:59.688784 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:00:59.689006 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:00:59.689007 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:00:59.689007 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:00:59.689346 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:00:59.689510 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0619 17:00:59.689635 (+ 125us) server_negotiation.cc:299] Negotiation successful
0619 17:00:59.689781 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":87,"threads_started":1}
I20260619 17:00:59.691160 21406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 17:00:59.707271 2615 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 17:00:59.708041 2615 catalog_manager.cc:1504] Loading token signing keys...
I20260619 17:00:59.714648 2615 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d4b2d8b65bf4eb8a6ea9fc70ca5e122: Generated new TSK 0
I20260619 17:00:59.714907 2615 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 17:00:59.745616 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 2528
2026-06-19T17:00:59Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1444 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 17:00:59 dist-test-slave-vnbj krb5kdc[2623](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 17:00:59 dist-test-slave-vnbj krb5kdc[2623](info): set up 2 sockets
Jun 19 17:00:59 dist-test-slave-vnbj krb5kdc[2623](info): commencing operation
krb5kdc: starting...
W20260619 17:01:00.839354 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 17:01:00 dist-test-slave-vnbj krb5kdc[2623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T17:01:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T17:01:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.231.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
I20260619 17:01:00.965685 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:33699
--webserver_interface=127.20.231.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:37209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:33699
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:01:01.048382 2636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:01:01.048594 2636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:01:01.048617 2636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:01:01.049856 2636 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:01:01.049906 2636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:01:01.049921 2636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:01:01.049933 2636 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:01:01.049973 2636 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:01:01.051481 2636 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:37209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:33699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:33699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:01:01.051687 2636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:01:01.051872 2636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:01:01.057139 2641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:01:01.057317 2644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:01:01.057421 2642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:01:01.057363 2636 server_base.cc:1034] running on GCE node
I20260619 17:01:01.057730 2636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:01:01.057979 2636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:01:01.059139 2636 hybrid_clock.cc:648] HybridClock initialized: now 1781888461059116 us; error 53 us; skew 500 ppm
Jun 19 17:01:01 dist-test-slave-vnbj krb5kdc[2623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888461, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:01:01.061918 2636 init.cc:377] Logged in from keytab as kudu/127.20.231.190@KRBTEST.COM (short username kudu)
I20260619 17:01:01.062430 2636 webserver.cc:460] Webserver started at http://127.20.231.190:34589/ using document root <none> and password file <none>
I20260619 17:01:01.062691 2636 fs_manager.cc:359] Metadata directory not provided
I20260619 17:01:01.062764 2636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:01:01.062951 2636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:01:01.064119 2636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "7b6203e4ca6b4fb7b12e8804e23fc087"
format_stamp: "Formatted at 2026-06-19 17:01:01 on dist-test-slave-vnbj"
server_key: "68f94e1b11156675e8acd0acbc1edca7"
server_key_iv: "905786f24e1770660bd75d86247322ea"
server_key_version: "encryptionkey@0"
I20260619 17:01:01.064498 2636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "7b6203e4ca6b4fb7b12e8804e23fc087"
format_stamp: "Formatted at 2026-06-19 17:01:01 on dist-test-slave-vnbj"
server_key: "68f94e1b11156675e8acd0acbc1edca7"
server_key_iv: "905786f24e1770660bd75d86247322ea"
server_key_version: "encryptionkey@0"
I20260619 17:01:01.065982 2636 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 17:01:01.067098 2651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:01.067303 2636 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 17:01:01.067431 2636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "7b6203e4ca6b4fb7b12e8804e23fc087"
format_stamp: "Formatted at 2026-06-19 17:01:01 on dist-test-slave-vnbj"
server_key: "68f94e1b11156675e8acd0acbc1edca7"
server_key_iv: "905786f24e1770660bd75d86247322ea"
server_key_version: "encryptionkey@0"
I20260619 17:01:01.067512 2636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:01:01.076476 2636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:01:01.078640 2636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:01:01.078801 2636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:01:01.082971 2636 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:33699
I20260619 17:01:01.083026 2703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:33699 every 8 connection(s)
I20260619 17:01:01.083405 2636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 17:01:01.084461 2704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:01:01.086755 2704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087: Bootstrap starting.
I20260619 17:01:01.087404 2704 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087: Neither blocks nor log segments found. Creating new log.
I20260619 17:01:01.087641 2704 log.cc:824] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087: Log is configured to *not* fsync() on all Append() calls
I20260619 17:01:01.088343 2704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087: No bootstrap required, opened a new log
I20260619 17:01:01.090029 2704 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6203e4ca6b4fb7b12e8804e23fc087" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 33699 } }
I20260619 17:01:01.090179 2704 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:01:01.090204 2704 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b6203e4ca6b4fb7b12e8804e23fc087, State: Initialized, Role: FOLLOWER
I20260619 17:01:01.090356 2704 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6203e4ca6b4fb7b12e8804e23fc087" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 33699 } }
I20260619 17:01:01.090443 2704 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 17:01:01.090466 2704 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 17:01:01.090494 2704 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:01:01.090767 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 2636
I20260619 17:01:01.090912 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 17:01:01.091118 21406 external_mini_cluster.cc:1428] Setting key 42d364313b3f4c5fc286fa869634f68d
I20260619 17:01:01.091727 2704 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6203e4ca6b4fb7b12e8804e23fc087" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 33699 } }
I20260619 17:01:01.091849 2704 leader_election.cc:302] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b6203e4ca6b4fb7b12e8804e23fc087; no voters:
I20260619 17:01:01.092046 2704 leader_election.cc:288] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 17:01:01.092062 2709 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:01:01.092171 2709 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [term 1 LEADER]: Becoming Leader. State: Replica: 7b6203e4ca6b4fb7b12e8804e23fc087, State: Running, Role: LEADER
I20260619 17:01:01.092267 2709 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6203e4ca6b4fb7b12e8804e23fc087" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 33699 } }
I20260619 17:01:01.092305 2704 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:01:01.092589 2711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b6203e4ca6b4fb7b12e8804e23fc087" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6203e4ca6b4fb7b12e8804e23fc087" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 33699 } } }
I20260619 17:01:01.092594 2710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b6203e4ca6b4fb7b12e8804e23fc087. Latest consensus state: current_term: 1 leader_uuid: "7b6203e4ca6b4fb7b12e8804e23fc087" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6203e4ca6b4fb7b12e8804e23fc087" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 33699 } } }
I20260619 17:01:01.092767 2711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [sys.catalog]: This master's current role is: LEADER
I20260619 17:01:01.092787 2710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [sys.catalog]: This master's current role is: LEADER
I20260619 17:01:01.093668 2718 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 17:01:01.094390 2718 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 17:01:01.096436 2718 catalog_manager.cc:1347] Generated new cluster ID: 4bc0593d767e445fa2edfb11df956b67
I20260619 17:01:01.096504 2718 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 17:01:01 dist-test-slave-vnbj krb5kdc[2623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:01:01.103242 2708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:01.091656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43074 (local address 127.20.231.190:33699)
0619 17:01:01.091842 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:01.091853 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:01.091940 (+ 87us) server_negotiation.cc:407] Connection header received
0619 17:01:01.092053 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:01.092059 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:01.092126 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:01.093093 (+ 967us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:01:01.093179 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.094638 (+ 1459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.095752 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.096259 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.096342 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:01.099300 (+ 2958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:01:01.099305 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:01:01.099305 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:01:01.099319 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:01:01.101408 (+ 2089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:01.101809 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:01.101813 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:01.101813 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:01.101942 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:01.102152 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:01.102153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:01.102154 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:01.102500 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:01:01.102732 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:01.102821 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 17:01:01.103020 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":109,"threads_started":1}
I20260619 17:01:01.126168 2718 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 17:01:01.126837 2718 catalog_manager.cc:1504] Loading token signing keys...
I20260619 17:01:01.130470 2718 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087: Generated new TSK 0
I20260619 17:01:01.130667 2718 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.231.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab.
Entry for principal kudu/127.20.231.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab.
I20260619 17:01:01.159758 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.129:0
--local_ip_for_outbound_sockets=127.20.231.129
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:33699
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:37209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:01:01.242293 2730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:01:01.242561 2730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:01:01.242600 2730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:01:01.243809 2730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:01:01.243877 2730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:01:01.243922 2730 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.129
I20260619 17:01:01.245473 2730 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:37209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:33699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2730
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.129
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:01:01.245759 2730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:01:01.246001 2730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:01:01.251194 2736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:01:01.251220 2730 server_base.cc:1034] running on GCE node
W20260619 17:01:01.251200 2735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:01:01.251204 2738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:01:01.251710 2730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:01:01.251978 2730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:01:01.253153 2730 hybrid_clock.cc:648] HybridClock initialized: now 1781888461253134 us; error 49 us; skew 500 ppm
Jun 19 17:01:01 dist-test-slave-vnbj krb5kdc[2623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888461, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:01:01.255754 2730 init.cc:377] Logged in from keytab as kudu/127.20.231.129@KRBTEST.COM (short username kudu)
I20260619 17:01:01.256183 2730 webserver.cc:460] Webserver started at http://127.20.231.129:44057/ using document root <none> and password file <none>
I20260619 17:01:01.256402 2730 fs_manager.cc:359] Metadata directory not provided
I20260619 17:01:01.256448 2730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:01:01.256569 2730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:01:01.257638 2730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/data/instance:
uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4"
format_stamp: "Formatted at 2026-06-19 17:01:01 on dist-test-slave-vnbj"
server_key: "0342c76ec53e01a9a502777e24968d93"
server_key_iv: "c3fb1de100370d6adf7f8d2a2fc0a550"
server_key_version: "encryptionkey@0"
I20260619 17:01:01.257999 2730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance:
uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4"
format_stamp: "Formatted at 2026-06-19 17:01:01 on dist-test-slave-vnbj"
server_key: "0342c76ec53e01a9a502777e24968d93"
server_key_iv: "c3fb1de100370d6adf7f8d2a2fc0a550"
server_key_version: "encryptionkey@0"
I20260619 17:01:01.259418 2730 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 17:01:01.260334 2745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:01.260592 2730 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:01:01.260680 2730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/wal
uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4"
format_stamp: "Formatted at 2026-06-19 17:01:01 on dist-test-slave-vnbj"
server_key: "0342c76ec53e01a9a502777e24968d93"
server_key_iv: "c3fb1de100370d6adf7f8d2a2fc0a550"
server_key_version: "encryptionkey@0"
I20260619 17:01:01.260753 2730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:01:01.271210 2730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:01:01.273319 2730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:01:01.273456 2730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:01:01.273698 2730 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:01:01.274019 2730 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:01:01.274070 2730 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:01.274113 2730 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:01:01.274138 2730 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:01.280040 2730 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.129:38807
I20260619 17:01:01.280095 2858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.129:38807 every 8 connection(s)
I20260619 17:01:01.280400 2730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
I20260619 17:01:01.284895 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 2730
I20260619 17:01:01.285012 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance
I20260619 17:01:01.285199 21406 external_mini_cluster.cc:1428] Setting key 2968ed44ef142b838f285d540ebca7b9
Jun 19 17:01:01 dist-test-slave-vnbj krb5kdc[2623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888461, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:01:01.290742 2861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:01.281090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.129:47149)
0619 17:01:01.281265 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:01.281316 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 17:01:01.281854 (+ 538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:01:01.282096 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:01:01.282099 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:01:01.282684 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:01:01.282948 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:01.282954 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.283665 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.283666 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:01.284470 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:01.284473 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.284993 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.284995 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:01.285150 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:01.285732 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:01:01.285812 (+ 80us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:01:01.287512 (+ 1700us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:01:01.289299 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:01:01.289305 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:01:01.289306 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:01:01.289534 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:01:01.289754 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:01:01.289755 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:01:01.289756 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:01:01.289929 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:01:01.290242 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:01:01.290248 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:01:01.290366 (+ 118us) client_negotiation.cc:769] Sending connection context
0619 17:01:01.290454 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 17:01:01.290579 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":87,"threads_started":1}
I20260619 17:01:01.290736 2708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:01.281140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:47149 (local address 127.20.231.190:33699)
0619 17:01:01.281225 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:01.281227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:01.281431 (+ 204us) server_negotiation.cc:407] Connection header received
0619 17:01:01.281960 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:01.281961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:01.281976 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:01.282030 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:01:01.283062 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.283574 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.284580 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.284848 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.284935 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:01.287608 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:01:01.287610 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:01:01.287611 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:01:01.287619 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:01:01.289207 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:01.289614 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:01.289616 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:01.289616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:01.289679 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:01.290003 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:01.290004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:01.290005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:01.290156 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:01:01.290230 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:01.290504 (+ 274us) server_negotiation.cc:299] Negotiation successful
0619 17:01:01.290598 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 17:01:01.291337 2859 heartbeater.cc:347] Connected to a master server at 127.20.231.190:33699
I20260619 17:01:01.291450 2859 heartbeater.cc:464] Registering TS with master...
I20260619 17:01:01.291682 2859 heartbeater.cc:511] Master 127.20.231.190:33699 requested a full tablet report, sending...
I20260619 17:01:01.292213 2668 ts_manager.cc:194] Registered new tserver with Master: 82b7a7a6f8ce4fa798108c74c2b99ab4 (127.20.231.129:38807)
I20260619 17:01:01.293028 2668 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.129@KRBTEST.COM'} at 127.20.231.129:47149
WARNING: no policy specified for kudu/127.20.231.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab.
Entry for principal kudu/127.20.231.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab.
I20260619 17:01:01.313786 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.130:0
--local_ip_for_outbound_sockets=127.20.231.130
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:33699
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:37209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:01:01.396139 2864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:01:01.396350 2864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:01:01.396374 2864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:01:01.397568 2864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:01:01.397612 2864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:01:01.397639 2864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.130
I20260619 17:01:01.399209 2864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:37209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:33699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2864
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.130
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:01:01.399466 2864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:01:01.399699 2864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:01:01.405062 2872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:01:01.405062 2869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:01:01.405175 2864 server_base.cc:1034] running on GCE node
W20260619 17:01:01.405138 2870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:01:01.405519 2864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:01:01.405776 2864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:01:01.406908 2864 hybrid_clock.cc:648] HybridClock initialized: now 1781888461406888 us; error 39 us; skew 500 ppm
Jun 19 17:01:01 dist-test-slave-vnbj krb5kdc[2623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888461, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:01:01.412561 2864 init.cc:377] Logged in from keytab as kudu/127.20.231.130@KRBTEST.COM (short username kudu)
I20260619 17:01:01.413043 2864 webserver.cc:460] Webserver started at http://127.20.231.130:41793/ using document root <none> and password file <none>
I20260619 17:01:01.413278 2864 fs_manager.cc:359] Metadata directory not provided
I20260619 17:01:01.413345 2864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:01:01.413508 2864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:01:01.414597 2864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/data/instance:
uuid: "86ed8d3ae791430d8acf249777ed79e1"
format_stamp: "Formatted at 2026-06-19 17:01:01 on dist-test-slave-vnbj"
server_key: "363d46d09da75bca68799764d9f19455"
server_key_iv: "ff210e9fefa1a5a5779cb09c9bd7eee9"
server_key_version: "encryptionkey@0"
I20260619 17:01:01.415005 2864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance:
uuid: "86ed8d3ae791430d8acf249777ed79e1"
format_stamp: "Formatted at 2026-06-19 17:01:01 on dist-test-slave-vnbj"
server_key: "363d46d09da75bca68799764d9f19455"
server_key_iv: "ff210e9fefa1a5a5779cb09c9bd7eee9"
server_key_version: "encryptionkey@0"
I20260619 17:01:01.416518 2864 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 17:01:01.417272 2879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:01.417454 2864 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:01.417557 2864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/wal
uuid: "86ed8d3ae791430d8acf249777ed79e1"
format_stamp: "Formatted at 2026-06-19 17:01:01 on dist-test-slave-vnbj"
server_key: "363d46d09da75bca68799764d9f19455"
server_key_iv: "ff210e9fefa1a5a5779cb09c9bd7eee9"
server_key_version: "encryptionkey@0"
I20260619 17:01:01.417639 2864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:01:01.444803 2864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:01:01.447217 2864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:01:01.447391 2864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:01:01.447641 2864 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:01:01.447964 2864 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:01:01.448021 2864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:01.448065 2864 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:01:01.448091 2864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:01.454202 2864 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.130:42577
I20260619 17:01:01.454241 2992 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.130:42577 every 8 connection(s)
I20260619 17:01:01.454620 2864 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
I20260619 17:01:01.459051 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 2864
I20260619 17:01:01.459174 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance
I20260619 17:01:01.459360 21406 external_mini_cluster.cc:1428] Setting key 1c176cfab78d71e04253bd4ef3dbbe7f
Jun 19 17:01:01 dist-test-slave-vnbj krb5kdc[2623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888461, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:01:01.465425 2708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:01.455447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:52059 (local address 127.20.231.190:33699)
0619 17:01:01.455514 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:01.455516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:01.455698 (+ 182us) server_negotiation.cc:407] Connection header received
0619 17:01:01.456248 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:01.456250 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:01.456260 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:01.456310 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:01:01.457208 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.457703 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.458693 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.459040 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.459112 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:01.461997 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:01:01.461999 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:01:01.461999 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:01:01.462007 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:01:01.463854 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:01.464254 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:01.464256 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:01.464256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:01.464330 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:01.464679 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:01.464680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:01.464680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:01.464849 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:01:01.464937 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:01.465200 (+ 263us) server_negotiation.cc:299] Negotiation successful
0619 17:01:01.465303 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 17:01:01.465457 2995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:01.455419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.130:52059)
0619 17:01:01.455573 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:01.455616 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 17:01:01.456162 (+ 546us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:01:01.456363 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:01:01.456366 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:01:01.456934 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:01:01.457126 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:01.457130 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.457786 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.457787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:01.458598 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:01.458599 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.459136 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.459138 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:01.459303 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:01.459947 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:01:01.459957 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:01:01.461893 (+ 1936us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:01:01.463947 (+ 2054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:01:01.463953 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:01:01.463955 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:01:01.464173 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:01:01.464425 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:01:01.464426 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:01:01.464427 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:01:01.464610 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:01:01.464928 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:01:01.464932 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:01:01.465053 (+ 121us) client_negotiation.cc:769] Sending connection context
0619 17:01:01.465140 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 17:01:01.465267 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":85,"threads_started":1}
I20260619 17:01:01.465963 2993 heartbeater.cc:347] Connected to a master server at 127.20.231.190:33699
I20260619 17:01:01.466092 2993 heartbeater.cc:464] Registering TS with master...
I20260619 17:01:01.466336 2993 heartbeater.cc:511] Master 127.20.231.190:33699 requested a full tablet report, sending...
I20260619 17:01:01.466781 2668 ts_manager.cc:194] Registered new tserver with Master: 86ed8d3ae791430d8acf249777ed79e1 (127.20.231.130:42577)
I20260619 17:01:01.467195 2668 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.130@KRBTEST.COM'} at 127.20.231.130:52059
WARNING: no policy specified for kudu/127.20.231.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab.
Entry for principal kudu/127.20.231.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab.
I20260619 17:01:01.491672 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.131:0
--local_ip_for_outbound_sockets=127.20.231.131
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:33699
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:37209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:01:01.577000 2998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:01:01.577214 2998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:01:01.577240 2998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:01:01.578433 2998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:01:01.578480 2998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:01:01.578507 2998 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.131
I20260619 17:01:01.580163 2998 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:37209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:33699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.131
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:01:01.580457 2998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:01:01.580724 2998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:01:01.585743 3003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:01:01.585891 3004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:01:01.585743 3006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:01:01.586103 2998 server_base.cc:1034] running on GCE node
I20260619 17:01:01.586318 2998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:01:01.586601 2998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:01:01.587760 2998 hybrid_clock.cc:648] HybridClock initialized: now 1781888461587703 us; error 63 us; skew 500 ppm
Jun 19 17:01:01 dist-test-slave-vnbj krb5kdc[2623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888461, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:01:01.590464 2998 init.cc:377] Logged in from keytab as kudu/127.20.231.131@KRBTEST.COM (short username kudu)
I20260619 17:01:01.591034 2998 webserver.cc:460] Webserver started at http://127.20.231.131:32951/ using document root <none> and password file <none>
I20260619 17:01:01.591267 2998 fs_manager.cc:359] Metadata directory not provided
I20260619 17:01:01.591336 2998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:01:01.591504 2998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:01:01.592690 2998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/data/instance:
uuid: "14686394078f408d8f2970b5a0218323"
format_stamp: "Formatted at 2026-06-19 17:01:01 on dist-test-slave-vnbj"
server_key: "8ae8083cb7a9b1783a580038e77b6d49"
server_key_iv: "78a34c3b8a3f7c1b78621f8d6f1f02a1"
server_key_version: "encryptionkey@0"
I20260619 17:01:01.593088 2998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance:
uuid: "14686394078f408d8f2970b5a0218323"
format_stamp: "Formatted at 2026-06-19 17:01:01 on dist-test-slave-vnbj"
server_key: "8ae8083cb7a9b1783a580038e77b6d49"
server_key_iv: "78a34c3b8a3f7c1b78621f8d6f1f02a1"
server_key_version: "encryptionkey@0"
I20260619 17:01:01.594569 2998 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 17:01:01.595450 3013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:01.595633 2998 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 17:01:01.595732 2998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/wal
uuid: "14686394078f408d8f2970b5a0218323"
format_stamp: "Formatted at 2026-06-19 17:01:01 on dist-test-slave-vnbj"
server_key: "8ae8083cb7a9b1783a580038e77b6d49"
server_key_iv: "78a34c3b8a3f7c1b78621f8d6f1f02a1"
server_key_version: "encryptionkey@0"
I20260619 17:01:01.595822 2998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:01:01.623893 2998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:01:01.625844 2998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:01:01.626015 2998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:01:01.626255 2998 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:01:01.626601 2998 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:01:01.626658 2998 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:01.626698 2998 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:01:01.626726 2998 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:01.633303 2998 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.131:36657
I20260619 17:01:01.633422 3126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.131:36657 every 8 connection(s)
I20260619 17:01:01.633749 2998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
I20260619 17:01:01.637176 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 2998
I20260619 17:01:01.637269 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance
I20260619 17:01:01.637432 21406 external_mini_cluster.cc:1428] Setting key a0c222169d839b5210722a12cd514763
Jun 19 17:01:01 dist-test-slave-vnbj krb5kdc[2623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888461, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:01:01.645471 2708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:01.634504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:56989 (local address 127.20.231.190:33699)
0619 17:01:01.634594 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:01.634597 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:01.634790 (+ 193us) server_negotiation.cc:407] Connection header received
0619 17:01:01.635388 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:01.635390 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:01.635404 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:01.635504 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:01:01.637011 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.637756 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.639977 (+ 2221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.640295 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.640366 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:01.642964 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:01:01.642965 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:01:01.642966 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:01:01.642981 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:01:01.644244 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:01.644647 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:01.644648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:01.644649 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:01.644699 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:01.644925 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:01.644925 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:01.644925 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:01.645032 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:01:01.645083 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:01.645336 (+ 253us) server_negotiation.cc:299] Negotiation successful
0619 17:01:01.645402 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 17:01:01.645676 3129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:01.634502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.131:56989)
0619 17:01:01.634670 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:01.634714 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 17:01:01.635286 (+ 572us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:01:01.635605 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:01:01.635610 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:01:01.636527 (+ 917us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:01:01.636852 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:01.636865 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.638667 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.638668 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:01.639876 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:01.639880 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.640408 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.640409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:01.640579 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:01.641117 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:01:01.641127 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:01:01.642806 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:01:01.644319 (+ 1513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:01:01.644325 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:01:01.644326 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:01:01.644573 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:01:01.644754 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:01:01.644755 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:01:01.644755 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:01:01.644861 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:01:01.645088 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:01:01.645093 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:01:01.645214 (+ 121us) client_negotiation.cc:769] Sending connection context
0619 17:01:01.645293 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 17:01:01.645431 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":97,"threads_started":1}
I20260619 17:01:01.646237 3127 heartbeater.cc:347] Connected to a master server at 127.20.231.190:33699
I20260619 17:01:01.646343 3127 heartbeater.cc:464] Registering TS with master...
I20260619 17:01:01.646533 3127 heartbeater.cc:511] Master 127.20.231.190:33699 requested a full tablet report, sending...
I20260619 17:01:01.647035 2668 ts_manager.cc:194] Registered new tserver with Master: 14686394078f408d8f2970b5a0218323 (127.20.231.131:36657)
I20260619 17:01:01.647478 2668 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.131@KRBTEST.COM'} at 127.20.231.131:56989
I20260619 17:01:01.649428 21406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 17:01:01.656834 2708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:01.650830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43082 (local address 127.20.231.190:33699)
0619 17:01:01.650943 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:01.650945 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:01.651004 (+ 59us) server_negotiation.cc:407] Connection header received
0619 17:01:01.651047 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:01.651047 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:01.651056 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:01.651102 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:01:01.651605 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.652069 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.652755 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.652966 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.653023 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:01.654267 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:01:01.654268 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:01:01.654268 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:01:01.654275 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:01:01.655642 (+ 1367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:01.656035 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:01.656036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:01.656037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:01.656095 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:01.656267 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:01.656268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:01.656268 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:01.656414 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:01:01.656497 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:01.656586 (+ 89us) server_negotiation.cc:299] Negotiation successful
0619 17:01:01.656765 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260619 17:01:01.658597 2668 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43082:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 17:01:01.660523 2668 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 17:01:01.668009 3140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:01.663431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45052 (local address 127.20.231.131:36657)
0619 17:01:01.664041 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:01.664046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:01.664057 (+ 11us) server_negotiation.cc:407] Connection header received
0619 17:01:01.664102 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:01.664104 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:01.664122 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:01.664422 (+ 300us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:01:01.664491 (+ 69us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.665304 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.666925 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.667608 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.667707 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:01.667766 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:01.667887 (+ 121us) server_negotiation.cc:299] Negotiation successful
0619 17:01:01.667928 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":105,"threads_started":1}
I20260619 17:01:01.667989 3138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:01.663316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:36657 (local address 127.0.0.1:45052)
0619 17:01:01.663628 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:01.663641 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 17:01:01.663704 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:01:01.664255 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:01:01.664256 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:01:01.664259 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:01:01.664335 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:01.664339 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.665430 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.665431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:01.666796 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:01.666799 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.667709 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.667711 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:01.667779 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:01.667784 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 17:01:01.667870 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 17:01:01.667907 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":62,"thread_start_us":104,"threads_started":1}
I20260619 17:01:01.667994 3139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:01.663297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:42577 (local address 127.0.0.1:45026)
0619 17:01:01.663482 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:01.663524 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 17:01:01.663593 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:01:01.664196 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:01:01.664199 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:01:01.664202 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:01:01.664290 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:01.664293 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.665753 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.665754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:01.666690 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:01.666693 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.667673 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.667675 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:01.667764 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:01.667779 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 17:01:01.667828 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 17:01:01.667886 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":3,"thread_start_us":78,"threads_started":1}
I20260619 17:01:01.668309 3142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:01.663567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45026 (local address 127.20.231.130:42577)
0619 17:01:01.663911 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:01.663916 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:01.663928 (+ 12us) server_negotiation.cc:407] Connection header received
0619 17:01:01.663971 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:01.663973 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:01.663990 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:01.664373 (+ 383us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:01:01.664447 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.665596 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.666780 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.667563 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.668041 (+ 478us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:01.668103 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:01.668132 (+ 29us) server_negotiation.cc:299] Negotiation successful
0619 17:01:01.668223 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":127,"threads_started":1}
I20260619 17:01:01.668830 3141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:01.663538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:38807 (local address 127.0.0.1:41122)
0619 17:01:01.663672 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:01.663681 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 17:01:01.663720 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:01:01.664410 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:01:01.664411 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:01:01.664413 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:01:01.664458 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:01.664460 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.665726 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.665727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:01.666966 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:01.666969 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.668322 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.668323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:01.668633 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:01.668639 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 17:01:01.668710 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 17:01:01.668753 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":84,"thread_start_us":68,"threads_started":1}
I20260619 17:01:01.669148 3143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:01.663887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41122 (local address 127.20.231.129:38807)
0619 17:01:01.664142 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:01.664148 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:01.664158 (+ 10us) server_negotiation.cc:407] Connection header received
0619 17:01:01.664208 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:01.664210 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:01.664225 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:01.664302 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:01:01.664538 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.665619 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.667068 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.667577 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.667705 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:01.667767 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:01.669004 (+ 1237us) server_negotiation.cc:299] Negotiation successful
0619 17:01:01.669062 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":105,"threads_started":1}
I20260619 17:01:01.669064 2927 tablet_service.cc:1467] Processing CreateTablet for tablet 1ac8833cf699482a88682b9df5b89e84 (DEFAULT_TABLE table=test-table [id=0169a62a838744c48d291c2b86638d94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 17:01:01.669226 3061 tablet_service.cc:1467] Processing CreateTablet for tablet 1ac8833cf699482a88682b9df5b89e84 (DEFAULT_TABLE table=test-table [id=0169a62a838744c48d291c2b86638d94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 17:01:01.669533 2927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ac8833cf699482a88682b9df5b89e84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:01:01.669571 3061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ac8833cf699482a88682b9df5b89e84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:01:01.669795 2793 tablet_service.cc:1467] Processing CreateTablet for tablet 1ac8833cf699482a88682b9df5b89e84 (DEFAULT_TABLE table=test-table [id=0169a62a838744c48d291c2b86638d94]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 17:01:01.670135 2793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ac8833cf699482a88682b9df5b89e84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:01:01.672633 3144 tablet_bootstrap.cc:492] T 1ac8833cf699482a88682b9df5b89e84 P 14686394078f408d8f2970b5a0218323: Bootstrap starting.
I20260619 17:01:01.673141 3145 tablet_bootstrap.cc:492] T 1ac8833cf699482a88682b9df5b89e84 P 86ed8d3ae791430d8acf249777ed79e1: Bootstrap starting.
I20260619 17:01:01.673426 3144 tablet_bootstrap.cc:654] T 1ac8833cf699482a88682b9df5b89e84 P 14686394078f408d8f2970b5a0218323: Neither blocks nor log segments found. Creating new log.
I20260619 17:01:01.673813 3145 tablet_bootstrap.cc:654] T 1ac8833cf699482a88682b9df5b89e84 P 86ed8d3ae791430d8acf249777ed79e1: Neither blocks nor log segments found. Creating new log.
I20260619 17:01:01.673933 3146 tablet_bootstrap.cc:492] T 1ac8833cf699482a88682b9df5b89e84 P 82b7a7a6f8ce4fa798108c74c2b99ab4: Bootstrap starting.
I20260619 17:01:01.674043 3144 log.cc:824] T 1ac8833cf699482a88682b9df5b89e84 P 14686394078f408d8f2970b5a0218323: Log is configured to *not* fsync() on all Append() calls
I20260619 17:01:01.674419 3145 log.cc:824] T 1ac8833cf699482a88682b9df5b89e84 P 86ed8d3ae791430d8acf249777ed79e1: Log is configured to *not* fsync() on all Append() calls
I20260619 17:01:01.674676 3146 tablet_bootstrap.cc:654] T 1ac8833cf699482a88682b9df5b89e84 P 82b7a7a6f8ce4fa798108c74c2b99ab4: Neither blocks nor log segments found. Creating new log.
I20260619 17:01:01.674988 3144 tablet_bootstrap.cc:492] T 1ac8833cf699482a88682b9df5b89e84 P 14686394078f408d8f2970b5a0218323: No bootstrap required, opened a new log
I20260619 17:01:01.675066 3144 ts_tablet_manager.cc:1397] T 1ac8833cf699482a88682b9df5b89e84 P 14686394078f408d8f2970b5a0218323: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 17:01:01.675225 3145 tablet_bootstrap.cc:492] T 1ac8833cf699482a88682b9df5b89e84 P 86ed8d3ae791430d8acf249777ed79e1: No bootstrap required, opened a new log
I20260619 17:01:01.675285 3145 ts_tablet_manager.cc:1397] T 1ac8833cf699482a88682b9df5b89e84 P 86ed8d3ae791430d8acf249777ed79e1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 17:01:01.675381 3146 log.cc:824] T 1ac8833cf699482a88682b9df5b89e84 P 82b7a7a6f8ce4fa798108c74c2b99ab4: Log is configured to *not* fsync() on all Append() calls
I20260619 17:01:01.676105 3146 tablet_bootstrap.cc:492] T 1ac8833cf699482a88682b9df5b89e84 P 82b7a7a6f8ce4fa798108c74c2b99ab4: No bootstrap required, opened a new log
I20260619 17:01:01.676198 3146 ts_tablet_manager.cc:1397] T 1ac8833cf699482a88682b9df5b89e84 P 82b7a7a6f8ce4fa798108c74c2b99ab4: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260619 17:01:01.677085 3144 raft_consensus.cc:348] T 1ac8833cf699482a88682b9df5b89e84 P 14686394078f408d8f2970b5a0218323 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14686394078f408d8f2970b5a0218323" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36657 } } peers { permanent_uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 38807 } } peers { permanent_uuid: "86ed8d3ae791430d8acf249777ed79e1" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42577 } }
I20260619 17:01:01.677263 3144 raft_consensus.cc:374] T 1ac8833cf699482a88682b9df5b89e84 P 14686394078f408d8f2970b5a0218323 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:01:01.677188 3145 raft_consensus.cc:348] T 1ac8833cf699482a88682b9df5b89e84 P 86ed8d3ae791430d8acf249777ed79e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14686394078f408d8f2970b5a0218323" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36657 } } peers { permanent_uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 38807 } } peers { permanent_uuid: "86ed8d3ae791430d8acf249777ed79e1" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42577 } }
I20260619 17:01:01.677299 3144 raft_consensus.cc:729] T 1ac8833cf699482a88682b9df5b89e84 P 14686394078f408d8f2970b5a0218323 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14686394078f408d8f2970b5a0218323, State: Initialized, Role: FOLLOWER
I20260619 17:01:01.677336 3145 raft_consensus.cc:374] T 1ac8833cf699482a88682b9df5b89e84 P 86ed8d3ae791430d8acf249777ed79e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:01:01.677366 3145 raft_consensus.cc:729] T 1ac8833cf699482a88682b9df5b89e84 P 86ed8d3ae791430d8acf249777ed79e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86ed8d3ae791430d8acf249777ed79e1, State: Initialized, Role: FOLLOWER
I20260619 17:01:01.677466 3145 consensus_queue.cc:260] T 1ac8833cf699482a88682b9df5b89e84 P 86ed8d3ae791430d8acf249777ed79e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14686394078f408d8f2970b5a0218323" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36657 } } peers { permanent_uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 38807 } } peers { permanent_uuid: "86ed8d3ae791430d8acf249777ed79e1" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42577 } }
I20260619 17:01:01.677480 3144 consensus_queue.cc:260] T 1ac8833cf699482a88682b9df5b89e84 P 14686394078f408d8f2970b5a0218323 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14686394078f408d8f2970b5a0218323" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36657 } } peers { permanent_uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 38807 } } peers { permanent_uuid: "86ed8d3ae791430d8acf249777ed79e1" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42577 } }
I20260619 17:01:01.677748 3144 ts_tablet_manager.cc:1428] T 1ac8833cf699482a88682b9df5b89e84 P 14686394078f408d8f2970b5a0218323: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 17:01:01.677748 3145 ts_tablet_manager.cc:1428] T 1ac8833cf699482a88682b9df5b89e84 P 86ed8d3ae791430d8acf249777ed79e1: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 17:01:01.677871 2993 heartbeater.cc:503] Master 127.20.231.190:33699 was elected leader, sending a full tablet report...
I20260619 17:01:01.678138 3127 heartbeater.cc:503] Master 127.20.231.190:33699 was elected leader, sending a full tablet report...
I20260619 17:01:01.678251 3146 raft_consensus.cc:348] T 1ac8833cf699482a88682b9df5b89e84 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14686394078f408d8f2970b5a0218323" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36657 } } peers { permanent_uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 38807 } } peers { permanent_uuid: "86ed8d3ae791430d8acf249777ed79e1" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42577 } }
I20260619 17:01:01.678402 3146 raft_consensus.cc:374] T 1ac8833cf699482a88682b9df5b89e84 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:01:01.678437 3146 raft_consensus.cc:729] T 1ac8833cf699482a88682b9df5b89e84 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82b7a7a6f8ce4fa798108c74c2b99ab4, State: Initialized, Role: FOLLOWER
I20260619 17:01:01.678543 3146 consensus_queue.cc:260] T 1ac8833cf699482a88682b9df5b89e84 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14686394078f408d8f2970b5a0218323" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36657 } } peers { permanent_uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 38807 } } peers { permanent_uuid: "86ed8d3ae791430d8acf249777ed79e1" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42577 } }
I20260619 17:01:01.678867 3146 ts_tablet_manager.cc:1428] T 1ac8833cf699482a88682b9df5b89e84 P 82b7a7a6f8ce4fa798108c74c2b99ab4: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 17:01:01.679221 2859 heartbeater.cc:503] Master 127.20.231.190:33699 was elected leader, sending a full tablet report...
W20260619 17:01:01.705225 2994 tablet.cc:2369] T 1ac8833cf699482a88682b9df5b89e84 P 86ed8d3ae791430d8acf249777ed79e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 17:01:01.755033 3151 raft_consensus.cc:482] T 1ac8833cf699482a88682b9df5b89e84 P 86ed8d3ae791430d8acf249777ed79e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 17:01:01.755352 3151 raft_consensus.cc:504] T 1ac8833cf699482a88682b9df5b89e84 P 86ed8d3ae791430d8acf249777ed79e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14686394078f408d8f2970b5a0218323" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36657 } } peers { permanent_uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 38807 } } peers { permanent_uuid: "86ed8d3ae791430d8acf249777ed79e1" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42577 } }
I20260619 17:01:01.755851 3151 leader_election.cc:288] T 1ac8833cf699482a88682b9df5b89e84 P 86ed8d3ae791430d8acf249777ed79e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 14686394078f408d8f2970b5a0218323 (127.20.231.131:36657), 82b7a7a6f8ce4fa798108c74c2b99ab4 (127.20.231.129:38807)
I20260619 17:01:01.759892 3143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:01.756262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:51183 (local address 127.20.231.129:38807)
0619 17:01:01.756299 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:01.756302 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:01.756460 (+ 158us) server_negotiation.cc:407] Connection header received
0619 17:01:01.756513 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:01.756514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:01.756529 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:01.756597 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:01:01.756740 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.757435 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.758810 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.759458 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.759542 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:01.759550 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:01.759741 (+ 191us) server_negotiation.cc:299] Negotiation successful
0619 17:01:01.759820 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 17:01:01.759847 2995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:01.756046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:36657 (local address 127.20.231.130:51995)
0619 17:01:01.756096 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:01.756109 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 17:01:01.756200 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:01:01.756371 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:01:01.756372 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:01:01.756374 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:01:01.756453 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:01.756456 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.757423 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.757423 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:01.758673 (+ 1250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:01.758676 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.759557 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.759558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:01.759641 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:01.759642 (+ 1us) client_negotiation.cc:769] Sending connection context
0619 17:01:01.759703 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 17:01:01.759752 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24,"mutex_wait_us":30}
I20260619 17:01:01.759917 3140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:01.756175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:51995 (local address 127.20.231.131:36657)
0619 17:01:01.756233 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:01.756235 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:01.756247 (+ 12us) server_negotiation.cc:407] Connection header received
0619 17:01:01.756281 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:01.756282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:01.756293 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:01.756366 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:01:01.756536 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.757334 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.758827 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.759458 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.759541 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:01.759550 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:01.759803 (+ 253us) server_negotiation.cc:299] Negotiation successful
0619 17:01:01.759849 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 17:01:01.759855 3153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:01.756170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:38807 (local address 127.20.231.130:51183)
0619 17:01:01.756359 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:01.756368 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 17:01:01.756449 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:01:01.756615 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:01:01.756615 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:01:01.756617 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:01:01.756668 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:01.756669 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.757513 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.757514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:01.758676 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:01.758677 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.759557 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.759558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:01.759637 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:01.759641 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 17:01:01.759703 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 17:01:01.759751 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":12,"thread_start_us":131,"threads_started":1}
I20260619 17:01:01.760558 3081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ac8833cf699482a88682b9df5b89e84" candidate_uuid: "86ed8d3ae791430d8acf249777ed79e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14686394078f408d8f2970b5a0218323" is_pre_election: true
I20260619 17:01:01.760558 2813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ac8833cf699482a88682b9df5b89e84" candidate_uuid: "86ed8d3ae791430d8acf249777ed79e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4" is_pre_election: true
I20260619 17:01:01.760751 2813 raft_consensus.cc:2454] T 1ac8833cf699482a88682b9df5b89e84 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86ed8d3ae791430d8acf249777ed79e1 in term 0.
I20260619 17:01:01.760751 3081 raft_consensus.cc:2454] T 1ac8833cf699482a88682b9df5b89e84 P 14686394078f408d8f2970b5a0218323 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86ed8d3ae791430d8acf249777ed79e1 in term 0.
I20260619 17:01:01.761003 2883 leader_election.cc:302] T 1ac8833cf699482a88682b9df5b89e84 P 86ed8d3ae791430d8acf249777ed79e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82b7a7a6f8ce4fa798108c74c2b99ab4, 86ed8d3ae791430d8acf249777ed79e1; no voters:
I20260619 17:01:01.761229 3151 raft_consensus.cc:2781] T 1ac8833cf699482a88682b9df5b89e84 P 86ed8d3ae791430d8acf249777ed79e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 17:01:01.761320 3151 raft_consensus.cc:482] T 1ac8833cf699482a88682b9df5b89e84 P 86ed8d3ae791430d8acf249777ed79e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 17:01:01.761343 3151 raft_consensus.cc:3037] T 1ac8833cf699482a88682b9df5b89e84 P 86ed8d3ae791430d8acf249777ed79e1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:01:01.762176 3151 raft_consensus.cc:504] T 1ac8833cf699482a88682b9df5b89e84 P 86ed8d3ae791430d8acf249777ed79e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14686394078f408d8f2970b5a0218323" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36657 } } peers { permanent_uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 38807 } } peers { permanent_uuid: "86ed8d3ae791430d8acf249777ed79e1" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42577 } }
I20260619 17:01:01.762326 3151 leader_election.cc:288] T 1ac8833cf699482a88682b9df5b89e84 P 86ed8d3ae791430d8acf249777ed79e1 [CANDIDATE]: Term 1 election: Requested vote from peers 14686394078f408d8f2970b5a0218323 (127.20.231.131:36657), 82b7a7a6f8ce4fa798108c74c2b99ab4 (127.20.231.129:38807)
I20260619 17:01:01.762533 3081 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ac8833cf699482a88682b9df5b89e84" candidate_uuid: "86ed8d3ae791430d8acf249777ed79e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "14686394078f408d8f2970b5a0218323"
I20260619 17:01:01.762562 2813 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ac8833cf699482a88682b9df5b89e84" candidate_uuid: "86ed8d3ae791430d8acf249777ed79e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4"
I20260619 17:01:01.762625 3081 raft_consensus.cc:3037] T 1ac8833cf699482a88682b9df5b89e84 P 14686394078f408d8f2970b5a0218323 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:01:01.762655 2813 raft_consensus.cc:3037] T 1ac8833cf699482a88682b9df5b89e84 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:01:01.763526 3081 raft_consensus.cc:2454] T 1ac8833cf699482a88682b9df5b89e84 P 14686394078f408d8f2970b5a0218323 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86ed8d3ae791430d8acf249777ed79e1 in term 1.
I20260619 17:01:01.763543 2813 raft_consensus.cc:2454] T 1ac8833cf699482a88682b9df5b89e84 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86ed8d3ae791430d8acf249777ed79e1 in term 1.
I20260619 17:01:01.763732 2883 leader_election.cc:302] T 1ac8833cf699482a88682b9df5b89e84 P 86ed8d3ae791430d8acf249777ed79e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 82b7a7a6f8ce4fa798108c74c2b99ab4, 86ed8d3ae791430d8acf249777ed79e1; no voters:
I20260619 17:01:01.763851 3151 raft_consensus.cc:2781] T 1ac8833cf699482a88682b9df5b89e84 P 86ed8d3ae791430d8acf249777ed79e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:01:01.764027 3151 raft_consensus.cc:686] T 1ac8833cf699482a88682b9df5b89e84 P 86ed8d3ae791430d8acf249777ed79e1 [term 1 LEADER]: Becoming Leader. State: Replica: 86ed8d3ae791430d8acf249777ed79e1, State: Running, Role: LEADER
I20260619 17:01:01.764139 3151 consensus_queue.cc:237] T 1ac8833cf699482a88682b9df5b89e84 P 86ed8d3ae791430d8acf249777ed79e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14686394078f408d8f2970b5a0218323" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36657 } } peers { permanent_uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 38807 } } peers { permanent_uuid: "86ed8d3ae791430d8acf249777ed79e1" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42577 } }
I20260619 17:01:01.765107 2668 catalog_manager.cc:5495] T 1ac8833cf699482a88682b9df5b89e84 P 86ed8d3ae791430d8acf249777ed79e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 86ed8d3ae791430d8acf249777ed79e1 (127.20.231.130). New cstate: current_term: 1 leader_uuid: "86ed8d3ae791430d8acf249777ed79e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14686394078f408d8f2970b5a0218323" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 38807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "86ed8d3ae791430d8acf249777ed79e1" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42577 } health_report { overall_health: HEALTHY } } }
W20260619 17:01:01.781203 2860 tablet.cc:2369] T 1ac8833cf699482a88682b9df5b89e84 P 82b7a7a6f8ce4fa798108c74c2b99ab4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 17:01:01.785858 3142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:01.783424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45028 (local address 127.20.231.130:42577)
0619 17:01:01.783498 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:01.783500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:01.783508 (+ 8us) server_negotiation.cc:407] Connection header received
0619 17:01:01.783544 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:01.783545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:01.783564 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:01.783627 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 17:01:01.783751 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.784297 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.785037 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:01.785227 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:01.785283 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:01.785444 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 17:01:01.785551 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 17:01:01.785617 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:01.785652 (+ 35us) server_negotiation.cc:299] Negotiation successful
0619 17:01:01.785670 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 17:01:01.787873 3081 raft_consensus.cc:1261] T 1ac8833cf699482a88682b9df5b89e84 P 14686394078f408d8f2970b5a0218323 [term 1 FOLLOWER]: Refusing update from remote peer 86ed8d3ae791430d8acf249777ed79e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 17:01:01.787884 2813 raft_consensus.cc:1261] T 1ac8833cf699482a88682b9df5b89e84 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [term 1 FOLLOWER]: Refusing update from remote peer 86ed8d3ae791430d8acf249777ed79e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 17:01:01.788269 3151 consensus_queue.cc:1035] T 1ac8833cf699482a88682b9df5b89e84 P 86ed8d3ae791430d8acf249777ed79e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "14686394078f408d8f2970b5a0218323" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 17:01:01.788421 3154 consensus_queue.cc:1035] T 1ac8833cf699482a88682b9df5b89e84 P 86ed8d3ae791430d8acf249777ed79e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 17:01:01.791644 3158 mvcc.cc:205] Tried to move back new op lower bound from 7298615139480162304 to 7298615139386642432. Current Snapshot: MvccSnapshot[applied={T|T < 7298615139480162304}]
I20260619 17:01:01.792305 3157 mvcc.cc:205] Tried to move back new op lower bound from 7298615139480162304 to 7298615139386642432. Current Snapshot: MvccSnapshot[applied={T|T < 7298615139480162304 or (T in {7298615139480162304})}]
I20260619 17:01:01.792464 3159 mvcc.cc:205] Tried to move back new op lower bound from 7298615139480162304 to 7298615139386642432. Current Snapshot: MvccSnapshot[applied={T|T < 7298615139480162304}]
I20260619 17:01:01.794072 2667 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 17:01:01.794181 2667 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 17:01:01.795197 2667 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087: Sending DeleteTablet for 3 replicas of tablet 1ac8833cf699482a88682b9df5b89e84
I20260619 17:01:01.795559 2927 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ac8833cf699482a88682b9df5b89e84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:01:01 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:45026
I20260619 17:01:01.795598 2793 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ac8833cf699482a88682b9df5b89e84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:01:01 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:41122
I20260619 17:01:01.795822 3061 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ac8833cf699482a88682b9df5b89e84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:01:01 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:45052
I20260619 17:01:01.795948 3167 tablet_replica.cc:330] T 1ac8833cf699482a88682b9df5b89e84 P 82b7a7a6f8ce4fa798108c74c2b99ab4: stopping tablet replica
I20260619 17:01:01.795948 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 2730
I20260619 17:01:01.796852 3168 tablet_replica.cc:330] T 1ac8833cf699482a88682b9df5b89e84 P 14686394078f408d8f2970b5a0218323: stopping tablet replica
I20260619 17:01:01.796972 3168 raft_consensus.cc:2229] T 1ac8833cf699482a88682b9df5b89e84 P 14686394078f408d8f2970b5a0218323 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 17:01:01.797093 3168 raft_consensus.cc:2258] T 1ac8833cf699482a88682b9df5b89e84 P 14686394078f408d8f2970b5a0218323 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 17:01:01.797353 3166 tablet_replica.cc:330] T 1ac8833cf699482a88682b9df5b89e84 P 86ed8d3ae791430d8acf249777ed79e1: stopping tablet replica
I20260619 17:01:01.797457 3166 raft_consensus.cc:2229] T 1ac8833cf699482a88682b9df5b89e84 P 86ed8d3ae791430d8acf249777ed79e1 [term 1 LEADER]: Raft consensus shutting down.
I20260619 17:01:01.797592 3166 raft_consensus.cc:2258] T 1ac8833cf699482a88682b9df5b89e84 P 86ed8d3ae791430d8acf249777ed79e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 17:01:01.797637 3168 ts_tablet_manager.cc:1905] T 1ac8833cf699482a88682b9df5b89e84 P 14686394078f408d8f2970b5a0218323: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 17:01:01.798084 3166 ts_tablet_manager.cc:1905] T 1ac8833cf699482a88682b9df5b89e84 P 86ed8d3ae791430d8acf249777ed79e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 17:01:01.800127 3168 ts_tablet_manager.cc:1918] T 1ac8833cf699482a88682b9df5b89e84 P 14686394078f408d8f2970b5a0218323: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 17:01:01.800168 3166 ts_tablet_manager.cc:1918] T 1ac8833cf699482a88682b9df5b89e84 P 86ed8d3ae791430d8acf249777ed79e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 17:01:01.800232 3168 log.cc:1196] T 1ac8833cf699482a88682b9df5b89e84 P 14686394078f408d8f2970b5a0218323: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/wal/wals/1ac8833cf699482a88682b9df5b89e84
I20260619 17:01:01.800254 3166 log.cc:1196] T 1ac8833cf699482a88682b9df5b89e84 P 86ed8d3ae791430d8acf249777ed79e1: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/wal/wals/1ac8833cf699482a88682b9df5b89e84
I20260619 17:01:01.800571 3166 ts_tablet_manager.cc:1939] T 1ac8833cf699482a88682b9df5b89e84 P 86ed8d3ae791430d8acf249777ed79e1: Deleting consensus metadata
I20260619 17:01:01.800575 3168 ts_tablet_manager.cc:1939] T 1ac8833cf699482a88682b9df5b89e84 P 14686394078f408d8f2970b5a0218323: Deleting consensus metadata
I20260619 17:01:01.801064 2655 catalog_manager.cc:4841] TS 14686394078f408d8f2970b5a0218323 (127.20.231.131:36657): tablet 1ac8833cf699482a88682b9df5b89e84 (table test-table [id=0169a62a838744c48d291c2b86638d94]) successfully deleted
I20260619 17:01:01.801064 2653 catalog_manager.cc:4841] TS 86ed8d3ae791430d8acf249777ed79e1 (127.20.231.130:42577): tablet 1ac8833cf699482a88682b9df5b89e84 (table test-table [id=0169a62a838744c48d291c2b86638d94]) successfully deleted
W20260619 17:01:01.803586 2655 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.231.129:38807 (error 108)
I20260619 17:01:01.803778 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 2864
I20260619 17:01:01.805682 3141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:01.805498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:38807 (local address 127.0.0.1:41136)
0619 17:01:01.805586 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:01.805648 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.129:38807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260619 17:01:01.805899 2655 catalog_manager.cc:4584] TS 82b7a7a6f8ce4fa798108c74c2b99ab4 (127.20.231.129:38807): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1ac8833cf699482a88682b9df5b89e84: Network error: Client connection negotiation failed: client connection to 127.20.231.129:38807: connect: Connection refused (error 111)
I20260619 17:01:01.809078 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 2998
I20260619 17:01:01.813464 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 2636
I20260619 17:01:01.819317 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:33699
--webserver_interface=127.20.231.190
--webserver_port=34589
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:37209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:33699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:01:01.905457 3170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:01:01.905777 3170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:01:01.905823 3170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:01:01.907189 3170 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:01:01.907279 3170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:01:01.907305 3170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:01:01.907330 3170 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:01:01.907363 3170 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:01:01.908987 3170 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:37209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:33699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:33699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=34589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:01:01.909322 3170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:01:01.909605 3170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:01:01.919426 3179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:01:01.919456 3177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:01:01.919426 3176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:01:01.919608 3170 server_base.cc:1034] running on GCE node
I20260619 17:01:01.920053 3170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:01:01.920411 3170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:01:01.921607 3170 hybrid_clock.cc:648] HybridClock initialized: now 1781888461921586 us; error 37 us; skew 500 ppm
Jun 19 17:01:01 dist-test-slave-vnbj krb5kdc[2623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888461, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:01:01.924466 3170 init.cc:377] Logged in from keytab as kudu/127.20.231.190@KRBTEST.COM (short username kudu)
I20260619 17:01:01.924899 3170 webserver.cc:460] Webserver started at http://127.20.231.190:34589/ using document root <none> and password file <none>
I20260619 17:01:01.925161 3170 fs_manager.cc:359] Metadata directory not provided
I20260619 17:01:01.925225 3170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:01:01.926705 3170 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 17:01:01.927528 3186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:01.927842 3170 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 17:01:01.927930 3170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "7b6203e4ca6b4fb7b12e8804e23fc087"
format_stamp: "Formatted at 2026-06-19 17:01:01 on dist-test-slave-vnbj"
server_key: "68f94e1b11156675e8acd0acbc1edca7"
server_key_iv: "905786f24e1770660bd75d86247322ea"
server_key_version: "encryptionkey@0"
I20260619 17:01:01.928396 3170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:01:01.936728 3170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:01:01.939227 3170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:01:01.939397 3170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:01:01.943434 3170 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:33699
I20260619 17:01:01.943509 3238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:33699 every 8 connection(s)
I20260619 17:01:01.943904 3170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 17:01:01.944453 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 3170
I20260619 17:01:01.944744 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.129:38807
--local_ip_for_outbound_sockets=127.20.231.129
--tserver_master_addrs=127.20.231.190:33699
--webserver_port=44057
--webserver_interface=127.20.231.129
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:37209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 17:01:01.945792 3239 sys_catalog.cc:263] Verifying existing consensus state
I20260619 17:01:01.946872 3239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087: Bootstrap starting.
I20260619 17:01:01.955564 3239 log.cc:824] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087: Log is configured to *not* fsync() on all Append() calls
I20260619 17:01:01.957705 3239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 17:01:01.958057 3239 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087: Bootstrap complete.
I20260619 17:01:01.960481 3239 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6203e4ca6b4fb7b12e8804e23fc087" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 33699 } }
I20260619 17:01:01.960731 3239 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b6203e4ca6b4fb7b12e8804e23fc087, State: Initialized, Role: FOLLOWER
I20260619 17:01:01.961010 3239 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6203e4ca6b4fb7b12e8804e23fc087" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 33699 } }
I20260619 17:01:01.961097 3239 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 17:01:01.961185 3239 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 17:01:01.961241 3239 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [term 1 FOLLOWER]: Advancing to term 2
I20260619 17:01:01.962745 3239 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6203e4ca6b4fb7b12e8804e23fc087" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 33699 } }
I20260619 17:01:01.962921 3239 leader_election.cc:302] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b6203e4ca6b4fb7b12e8804e23fc087; no voters:
I20260619 17:01:01.963172 3239 leader_election.cc:288] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 17:01:01.963234 3243 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 17:01:01.963378 3243 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [term 2 LEADER]: Becoming Leader. State: Replica: 7b6203e4ca6b4fb7b12e8804e23fc087, State: Running, Role: LEADER
I20260619 17:01:01.963456 3239 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:01:01.964164 3243 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6203e4ca6b4fb7b12e8804e23fc087" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 33699 } }
I20260619 17:01:01.964893 3244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b6203e4ca6b4fb7b12e8804e23fc087. Latest consensus state: current_term: 2 leader_uuid: "7b6203e4ca6b4fb7b12e8804e23fc087" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6203e4ca6b4fb7b12e8804e23fc087" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 33699 } } }
I20260619 17:01:01.964771 3245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7b6203e4ca6b4fb7b12e8804e23fc087" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6203e4ca6b4fb7b12e8804e23fc087" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 33699 } } }
I20260619 17:01:01.965039 3244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [sys.catalog]: This master's current role is: LEADER
I20260619 17:01:01.965057 3245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [sys.catalog]: This master's current role is: LEADER
I20260619 17:01:01.965274 3257 catalog_manager.cc:1259] Loaded cluster ID: 4bc0593d767e445fa2edfb11df956b67
I20260619 17:01:01.965328 3257 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087: loading cluster ID for follower catalog manager: success
W20260619 17:01:01.965596 3257 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087: acquiring CA information for follower catalog manager: Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG
W20260619 17:01:01.965644 3257 catalog_manager.cc:873] Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG: failed to prepare follower catalog manager, will retry
I20260619 17:01:01.965871 3260 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 17:01:01.966291 3260 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 17:01:01.966372 3260 catalog_manager.cc:1259] Loaded cluster ID: 4bc0593d767e445fa2edfb11df956b67
I20260619 17:01:01.966400 3260 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 17:01:01.966602 3260 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260619 17:01:01.971168 3260 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260619 17:01:01.973701 3260 catalog_manager.cc:1504] Loading token signing keys...
I20260619 17:01:01.973920 3260 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087: Loaded TSK: 0
I20260619 17:01:01.974123 3260 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 17:01:02.033872 3241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:01:02.034086 3241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:01:02.034112 3241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:01:02.035357 3241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:01:02.035410 3241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:01:02.035437 3241 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.129
I20260619 17:01:02.036912 3241 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:37209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.129:38807
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.231.129
--webserver_port=44057
--tserver_master_addrs=127.20.231.190:33699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.129
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:01:02.037166 3241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:01:02.037429 3241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:01:02.044028 3268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:01:02.044054 3270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:01:02.044067 3267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:01:02.044755 3241 server_base.cc:1034] running on GCE node
I20260619 17:01:02.044968 3241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:01:02.045253 3241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:01:02.046430 3241 hybrid_clock.cc:648] HybridClock initialized: now 1781888462046409 us; error 37 us; skew 500 ppm
Jun 19 17:01:02 dist-test-slave-vnbj krb5kdc[2623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888462, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:01:02.049091 3241 init.cc:377] Logged in from keytab as kudu/127.20.231.129@KRBTEST.COM (short username kudu)
I20260619 17:01:02.049662 3241 webserver.cc:460] Webserver started at http://127.20.231.129:44057/ using document root <none> and password file <none>
I20260619 17:01:02.049955 3241 fs_manager.cc:359] Metadata directory not provided
I20260619 17:01:02.050016 3241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:01:02.051587 3241 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 17:01:02.052352 3277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:02.052599 3241 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:01:02.052702 3241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/wal
uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4"
format_stamp: "Formatted at 2026-06-19 17:01:01 on dist-test-slave-vnbj"
server_key: "0342c76ec53e01a9a502777e24968d93"
server_key_iv: "c3fb1de100370d6adf7f8d2a2fc0a550"
server_key_version: "encryptionkey@0"
I20260619 17:01:02.053097 3241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:01:02.079277 3241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:01:02.081441 3241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:01:02.081622 3241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:01:02.081883 3241 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:01:02.082357 3284 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 17:01:02.084337 3241 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 17:01:02.084411 3241 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 17:01:02.084456 3241 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 17:01:02.085124 3241 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 17:01:02.085182 3241 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 17:01:02.085247 3284 tablet_bootstrap.cc:492] T 1ac8833cf699482a88682b9df5b89e84 P 82b7a7a6f8ce4fa798108c74c2b99ab4: Bootstrap starting.
I20260619 17:01:02.093549 3241 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.129:38807
I20260619 17:01:02.093587 3391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.129:38807 every 8 connection(s)
I20260619 17:01:02.094094 3241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
I20260619 17:01:02.097348 3284 log.cc:824] T 1ac8833cf699482a88682b9df5b89e84 P 82b7a7a6f8ce4fa798108c74c2b99ab4: Log is configured to *not* fsync() on all Append() calls
I20260619 17:01:02.101110 3284 tablet_bootstrap.cc:492] T 1ac8833cf699482a88682b9df5b89e84 P 82b7a7a6f8ce4fa798108c74c2b99ab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 17:01:02.101420 3284 tablet_bootstrap.cc:492] T 1ac8833cf699482a88682b9df5b89e84 P 82b7a7a6f8ce4fa798108c74c2b99ab4: Bootstrap complete.
I20260619 17:01:02.102578 3284 ts_tablet_manager.cc:1397] T 1ac8833cf699482a88682b9df5b89e84 P 82b7a7a6f8ce4fa798108c74c2b99ab4: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.005s
I20260619 17:01:02.103236 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 3241
I20260619 17:01:02.103569 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.130:42577
--local_ip_for_outbound_sockets=127.20.231.130
--tserver_master_addrs=127.20.231.190:33699
--webserver_port=41793
--webserver_interface=127.20.231.130
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:37209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 17:01:02.103897 3284 raft_consensus.cc:348] T 1ac8833cf699482a88682b9df5b89e84 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14686394078f408d8f2970b5a0218323" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36657 } } peers { permanent_uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 38807 } } peers { permanent_uuid: "86ed8d3ae791430d8acf249777ed79e1" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42577 } }
I20260619 17:01:02.104206 3284 raft_consensus.cc:729] T 1ac8833cf699482a88682b9df5b89e84 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82b7a7a6f8ce4fa798108c74c2b99ab4, State: Initialized, Role: FOLLOWER
Jun 19 17:01:02 dist-test-slave-vnbj krb5kdc[2623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888462, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:01:02.104374 3284 consensus_queue.cc:260] T 1ac8833cf699482a88682b9df5b89e84 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "14686394078f408d8f2970b5a0218323" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36657 } } peers { permanent_uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 38807 } } peers { permanent_uuid: "86ed8d3ae791430d8acf249777ed79e1" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42577 } }
I20260619 17:01:02.104671 3284 ts_tablet_manager.cc:1428] T 1ac8833cf699482a88682b9df5b89e84 P 82b7a7a6f8ce4fa798108c74c2b99ab4: Time spent starting tablet: real 0.002s user 0.003s sys 0.003s
I20260619 17:01:02.110446 3394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:02.094897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.129:47141)
0619 17:01:02.095104 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:02.095160 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 17:01:02.095735 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:01:02.097988 (+ 2253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:01:02.097993 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:01:02.098344 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:01:02.098592 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:02.098598 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.099982 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.099983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:02.100959 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:02.100963 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.101844 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.101846 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:02.101993 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:02.102676 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:01:02.102688 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:01:02.104800 (+ 2112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:01:02.108494 (+ 3694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:01:02.108501 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:01:02.108503 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:01:02.108750 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:01:02.109101 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:01:02.109102 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:01:02.109102 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:01:02.109208 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:01:02.109858 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:01:02.109865 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:01:02.109995 (+ 130us) client_negotiation.cc:769] Sending connection context
0619 17:01:02.110095 (+ 100us) client_negotiation.cc:240] Negotiation successful
0619 17:01:02.110288 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":103,"threads_started":1}
I20260619 17:01:02.110705 3395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:02.095176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:47141 (local address 127.20.231.190:33699)
0619 17:01:02.097429 (+ 2253us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:02.097441 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:02.097493 (+ 52us) server_negotiation.cc:407] Connection header received
0619 17:01:02.097686 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:02.097694 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:02.097765 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:02.097952 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:01:02.098751 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.099804 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.101090 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.101700 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.101853 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:02.104931 (+ 3078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:01:02.104937 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:01:02.104937 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:01:02.104953 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:01:02.108387 (+ 3434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:02.108851 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:02.108855 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:02.108856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:02.109009 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:02.109291 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:02.109293 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:02.109293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:02.109707 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:01:02.109846 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:02.110202 (+ 356us) server_negotiation.cc:299] Negotiation successful
0619 17:01:02.110389 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2188,"thread_start_us":68,"threads_started":1}
I20260619 17:01:02.111541 3392 heartbeater.cc:347] Connected to a master server at 127.20.231.190:33699
I20260619 17:01:02.111691 3392 heartbeater.cc:464] Registering TS with master...
I20260619 17:01:02.111943 3392 heartbeater.cc:511] Master 127.20.231.190:33699 requested a full tablet report, sending...
I20260619 17:01:02.112793 3203 ts_manager.cc:194] Registered new tserver with Master: 82b7a7a6f8ce4fa798108c74c2b99ab4 (127.20.231.129:38807)
I20260619 17:01:02.113932 3203 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.129@KRBTEST.COM'} at 127.20.231.129:47141
I20260619 17:01:02.114974 3392 heartbeater.cc:503] Master 127.20.231.190:33699 was elected leader, sending a full tablet report...
I20260619 17:01:02.123214 3399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:02.113391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:38807 (local address 127.0.0.1:41140)
0619 17:01:02.113607 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:02.113638 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 17:01:02.113695 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:01:02.119273 (+ 5578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:01:02.119276 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:01:02.119279 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:01:02.119352 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:02.119355 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.120923 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.120925 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:02.121771 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:02.121773 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.122933 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.122934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:02.123011 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:02.123023 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 17:01:02.123096 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 17:01:02.123148 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":99,"threads_started":1}
I20260619 17:01:02.123284 3400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:02.113492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41140 (local address 127.20.231.129:38807)
0619 17:01:02.119017 (+ 5525us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:02.119022 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:02.119040 (+ 18us) server_negotiation.cc:407] Connection header received
0619 17:01:02.119089 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:02.119091 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:02.119113 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:02.119234 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:01:02.119447 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.120758 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.121873 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.122764 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.122891 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:02.122960 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:02.123131 (+ 171us) server_negotiation.cc:299] Negotiation successful
0619 17:01:02.123182 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5477,"thread_start_us":100,"threads_started":1}
I20260619 17:01:02.123795 3326 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ac8833cf699482a88682b9df5b89e84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:01:01 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:41140
I20260619 17:01:02.124512 3401 tablet_replica.cc:330] stopping tablet replica
I20260619 17:01:02.124630 3401 raft_consensus.cc:2229] T 1ac8833cf699482a88682b9df5b89e84 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 17:01:02.124723 3401 raft_consensus.cc:2258] T 1ac8833cf699482a88682b9df5b89e84 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 17:01:02.125020 3401 ts_tablet_manager.cc:1905] T 1ac8833cf699482a88682b9df5b89e84 P 82b7a7a6f8ce4fa798108c74c2b99ab4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 17:01:02.127223 3401 ts_tablet_manager.cc:1918] T 1ac8833cf699482a88682b9df5b89e84 P 82b7a7a6f8ce4fa798108c74c2b99ab4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 17:01:02.127290 3401 log.cc:1196] T 1ac8833cf699482a88682b9df5b89e84 P 82b7a7a6f8ce4fa798108c74c2b99ab4: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/wal/wals/1ac8833cf699482a88682b9df5b89e84
I20260619 17:01:02.127547 3401 ts_tablet_manager.cc:1939] T 1ac8833cf699482a88682b9df5b89e84 P 82b7a7a6f8ce4fa798108c74c2b99ab4: Deleting consensus metadata
I20260619 17:01:02.128118 3190 catalog_manager.cc:4841] TS 82b7a7a6f8ce4fa798108c74c2b99ab4 (127.20.231.129:38807): tablet 1ac8833cf699482a88682b9df5b89e84 (table test-table [id=0169a62a838744c48d291c2b86638d94]) successfully deleted
W20260619 17:01:02.202498 3398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:01:02.202785 3398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:01:02.202832 3398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:01:02.204139 3398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:01:02.204231 3398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:01:02.204280 3398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.130
I20260619 17:01:02.205812 3398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:37209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.130:42577
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.231.130
--webserver_port=41793
--tserver_master_addrs=127.20.231.190:33699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.130
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:01:02.206110 3398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:01:02.206377 3398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:01:02.211810 3406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:01:02.211804 3407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:01:02.211970 3409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:01:02.212263 3398 server_base.cc:1034] running on GCE node
I20260619 17:01:02.212522 3398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:01:02.212782 3398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:01:02.213977 3398 hybrid_clock.cc:648] HybridClock initialized: now 1781888462213965 us; error 31 us; skew 500 ppm
Jun 19 17:01:02 dist-test-slave-vnbj krb5kdc[2623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888462, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:01:02.216682 3398 init.cc:377] Logged in from keytab as kudu/127.20.231.130@KRBTEST.COM (short username kudu)
I20260619 17:01:02.217182 3398 webserver.cc:460] Webserver started at http://127.20.231.130:41793/ using document root <none> and password file <none>
I20260619 17:01:02.217433 3398 fs_manager.cc:359] Metadata directory not provided
I20260619 17:01:02.217494 3398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:01:02.219035 3398 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:01:02.219750 3416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:02.220060 3398 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:01:02.220162 3398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/wal
uuid: "86ed8d3ae791430d8acf249777ed79e1"
format_stamp: "Formatted at 2026-06-19 17:01:01 on dist-test-slave-vnbj"
server_key: "363d46d09da75bca68799764d9f19455"
server_key_iv: "ff210e9fefa1a5a5779cb09c9bd7eee9"
server_key_version: "encryptionkey@0"
I20260619 17:01:02.220533 3398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:01:02.238770 3398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:01:02.241308 3398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:01:02.241431 3398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:01:02.241772 3398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:01:02.242125 3398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:01:02.242161 3398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:02.242183 3398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:01:02.242228 3398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:02.248245 3398 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.130:42577
I20260619 17:01:02.248344 3529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.130:42577 every 8 connection(s)
I20260619 17:01:02.248768 3398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
I20260619 17:01:02.250074 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 3398
I20260619 17:01:02.250368 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.131:36657
--local_ip_for_outbound_sockets=127.20.231.131
--tserver_master_addrs=127.20.231.190:33699
--webserver_port=32951
--webserver_interface=127.20.231.131
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:37209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 17:01:02 dist-test-slave-vnbj krb5kdc[2623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888462, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:01:02.260308 3532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:02.249531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.130:51335)
0619 17:01:02.249697 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:02.249749 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 17:01:02.250500 (+ 751us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:01:02.250811 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:01:02.250816 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:01:02.251712 (+ 896us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:01:02.252030 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:02.252038 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.252746 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.252748 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:02.254103 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:02.254106 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.254496 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.254497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:02.254659 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:02.255258 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:01:02.255270 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:01:02.257036 (+ 1766us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:01:02.258933 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:01:02.258940 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:01:02.258941 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:01:02.259171 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:01:02.259408 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:01:02.259408 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:01:02.259408 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:01:02.259505 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:01:02.259803 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:01:02.259807 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:01:02.259930 (+ 123us) client_negotiation.cc:769] Sending connection context
0619 17:01:02.260013 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 17:01:02.260137 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":95,"threads_started":1}
I20260619 17:01:02.260303 3395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:02.249567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:51335 (local address 127.20.231.190:33699)
0619 17:01:02.249625 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:02.249627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:02.249898 (+ 271us) server_negotiation.cc:407] Connection header received
0619 17:01:02.250636 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:02.250638 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:02.250659 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:02.250718 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:01:02.252139 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.252642 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.254196 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.254403 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.254461 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:02.257133 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:01:02.257135 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:01:02.257136 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:01:02.257145 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:01:02.258825 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:02.259255 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:02.259256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:02.259256 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:02.259324 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:02.259572 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:02.259573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:02.259574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:02.259734 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:01:02.259812 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:02.260060 (+ 248us) server_negotiation.cc:299] Negotiation successful
0619 17:01:02.260158 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260619 17:01:02.260876 3530 heartbeater.cc:347] Connected to a master server at 127.20.231.190:33699
I20260619 17:01:02.260999 3530 heartbeater.cc:464] Registering TS with master...
I20260619 17:01:02.261200 3530 heartbeater.cc:511] Master 127.20.231.190:33699 requested a full tablet report, sending...
I20260619 17:01:02.261616 3203 ts_manager.cc:194] Registered new tserver with Master: 86ed8d3ae791430d8acf249777ed79e1 (127.20.231.130:42577)
I20260619 17:01:02.261996 3203 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.130@KRBTEST.COM'} at 127.20.231.130:51335
W20260619 17:01:02.338151 3533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:01:02.338364 3533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:01:02.338388 3533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:01:02.339663 3533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:01:02.339708 3533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:01:02.339761 3533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.131
I20260619 17:01:02.341202 3533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:37209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.131:36657
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.231.131
--webserver_port=32951
--tserver_master_addrs=127.20.231.190:33699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.131
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:01:02.341457 3533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:01:02.341702 3533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:01:02.346657 3539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:01:02.346688 3538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:01:02.346707 3541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:01:02.347079 3533 server_base.cc:1034] running on GCE node
I20260619 17:01:02.347270 3533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:01:02.347502 3533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:01:02.348652 3533 hybrid_clock.cc:648] HybridClock initialized: now 1781888462348640 us; error 29 us; skew 500 ppm
Jun 19 17:01:02 dist-test-slave-vnbj krb5kdc[2623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888462, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:01:02.351409 3533 init.cc:377] Logged in from keytab as kudu/127.20.231.131@KRBTEST.COM (short username kudu)
I20260619 17:01:02.351850 3533 webserver.cc:460] Webserver started at http://127.20.231.131:32951/ using document root <none> and password file <none>
I20260619 17:01:02.352108 3533 fs_manager.cc:359] Metadata directory not provided
I20260619 17:01:02.352190 3533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:01:02.353603 3533 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:01:02.354236 3548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:02.354440 3533 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:01:02.354537 3533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/wal
uuid: "14686394078f408d8f2970b5a0218323"
format_stamp: "Formatted at 2026-06-19 17:01:01 on dist-test-slave-vnbj"
server_key: "8ae8083cb7a9b1783a580038e77b6d49"
server_key_iv: "78a34c3b8a3f7c1b78621f8d6f1f02a1"
server_key_version: "encryptionkey@0"
I20260619 17:01:02.354965 3533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:01:02.369948 3533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:01:02.372143 3533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:01:02.372313 3533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:01:02.372545 3533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:01:02.372852 3533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:01:02.372915 3533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:02.372956 3533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:01:02.372984 3533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:02.378945 3661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.131:36657 every 8 connection(s)
I20260619 17:01:02.378949 3533 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.131:36657
I20260619 17:01:02.379383 3533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
Jun 19 17:01:02 dist-test-slave-vnbj krb5kdc[2623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888462, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:01:02.386608 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 3533
I20260619 17:01:02.389146 3395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:02.380166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:49065 (local address 127.20.231.190:33699)
0619 17:01:02.380226 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:02.380228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:02.380447 (+ 219us) server_negotiation.cc:407] Connection header received
0619 17:01:02.381087 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:02.381089 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:02.381102 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:02.381148 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:01:02.382110 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.382613 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.383666 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.383873 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.383960 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:02.386100 (+ 2140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:01:02.386102 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:01:02.386102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:01:02.386110 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:01:02.387679 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:02.388170 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:02.388171 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:02.388171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:02.388226 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:02.388492 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:02.388493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:02.388493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:02.388594 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:01:02.388645 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:02.388975 (+ 330us) server_negotiation.cc:299] Negotiation successful
0619 17:01:02.389054 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 17:01:02.389256 3664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:02.380157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.131:49065)
0619 17:01:02.380324 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:02.380361 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0619 17:01:02.380955 (+ 594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:01:02.381223 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:01:02.381227 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:01:02.381819 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:01:02.382011 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:02.382016 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.382710 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.382711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:02.383574 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:02.383576 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.383970 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.383971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:02.384092 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:02.384564 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:01:02.384573 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:01:02.385999 (+ 1426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:01:02.387767 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:01:02.387774 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:01:02.387776 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:01:02.388088 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:01:02.388290 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:01:02.388291 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:01:02.388291 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:01:02.388425 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:01:02.388656 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:01:02.388661 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:01:02.388834 (+ 173us) client_negotiation.cc:769] Sending connection context
0619 17:01:02.388933 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 17:01:02.389051 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":79,"threads_started":1}
I20260619 17:01:02.389716 3662 heartbeater.cc:347] Connected to a master server at 127.20.231.190:33699
I20260619 17:01:02.389868 3662 heartbeater.cc:464] Registering TS with master...
I20260619 17:01:02.390055 3662 heartbeater.cc:511] Master 127.20.231.190:33699 requested a full tablet report, sending...
I20260619 17:01:02.390492 3203 ts_manager.cc:194] Registered new tserver with Master: 14686394078f408d8f2970b5a0218323 (127.20.231.131:36657)
I20260619 17:01:02.390998 3203 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.131@KRBTEST.COM'} at 127.20.231.131:49065
I20260619 17:01:02.394001 3666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:02.387079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43098 (local address 127.20.231.190:33699)
0619 17:01:02.387229 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:02.387231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:02.387239 (+ 8us) server_negotiation.cc:407] Connection header received
0619 17:01:02.387285 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:02.387285 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:02.387298 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:02.387360 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:01:02.388004 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.388728 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.389870 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.390182 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.390247 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:02.391592 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:01:02.391593 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:01:02.391594 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:01:02.391600 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:01:02.392887 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:02.393210 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:02.393211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:02.393211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:02.393288 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:02.393507 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:02.393508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:02.393508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:02.393622 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:01:02.393777 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:02.393811 (+ 34us) server_negotiation.cc:299] Negotiation successful
0619 17:01:02.393876 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":73,"threads_started":1}
I20260619 17:01:02.394776 21406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 17:01:02.397342 3666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:02.395113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43114 (local address 127.20.231.190:33699)
0619 17:01:02.395144 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:02.395145 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:02.395237 (+ 92us) server_negotiation.cc:407] Connection header received
0619 17:01:02.395291 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:02.395291 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:02.395315 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:02.395375 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 17:01:02.395455 (+ 80us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.395932 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.396649 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.396884 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.396952 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:02.397074 (+ 122us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 17:01:02.397164 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 17:01:02.397254 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:02.397274 (+ 20us) server_negotiation.cc:299] Negotiation successful
0619 17:01:02.397294 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260619 17:01:02.397610 3203 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:43114:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 17:01:02.399362 3203 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 17:01:02.401567 3326 tablet_service.cc:1467] Processing CreateTablet for tablet de40b7a929314c468cc3617774433870 (DEFAULT_TABLE table=test-table [id=d0713b9d199a441aa048b9ad78355b59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 17:01:02.401780 3326 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de40b7a929314c468cc3617774433870. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:01:02.403075 3284 tablet_bootstrap.cc:492] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4: Bootstrap starting.
I20260619 17:01:02.403420 3284 tablet_bootstrap.cc:654] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4: Neither blocks nor log segments found. Creating new log.
I20260619 17:01:02.403983 3284 tablet_bootstrap.cc:492] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4: No bootstrap required, opened a new log
I20260619 17:01:02.404049 3284 ts_tablet_manager.cc:1397] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260619 17:01:02.404166 3284 raft_consensus.cc:348] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86ed8d3ae791430d8acf249777ed79e1" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42577 } } peers { permanent_uuid: "14686394078f408d8f2970b5a0218323" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36657 } } peers { permanent_uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 38807 } }
I20260619 17:01:02.404256 3284 raft_consensus.cc:374] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:01:02.404313 3284 raft_consensus.cc:729] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82b7a7a6f8ce4fa798108c74c2b99ab4, State: Initialized, Role: FOLLOWER
I20260619 17:01:02.404361 3284 consensus_queue.cc:260] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86ed8d3ae791430d8acf249777ed79e1" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42577 } } peers { permanent_uuid: "14686394078f408d8f2970b5a0218323" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36657 } } peers { permanent_uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 38807 } }
I20260619 17:01:02.404464 3284 ts_tablet_manager.cc:1428] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:02.405550 3668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:02.401132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:36657 (local address 127.0.0.1:45064)
0619 17:01:02.401320 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:02.401328 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 17:01:02.401368 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:01:02.401653 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:01:02.401654 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:01:02.401656 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:01:02.401703 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:02.401705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.403129 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.403129 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:02.404300 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:02.404303 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.405297 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.405297 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:02.405361 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:02.405366 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 17:01:02.405415 (+ 49us) client_negotiation.cc:240] Negotiation successful
0619 17:01:02.405460 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":21,"thread_start_us":97,"threads_started":1}
I20260619 17:01:02.405638 3399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:02.401132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:42577 (local address 127.0.0.1:45034)
0619 17:01:02.401189 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:02.401201 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 17:01:02.401237 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:01:02.401963 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:01:02.401964 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:01:02.401965 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:01:02.401996 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:02.401997 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.403312 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.403313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:02.404335 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:02.404337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.405452 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.405453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:02.405504 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:02.405507 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 17:01:02.405534 (+ 27us) client_negotiation.cc:240] Negotiation successful
0619 17:01:02.405573 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 17:01:02.405808 3669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:02.401252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45064 (local address 127.20.231.131:36657)
0619 17:01:02.401507 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:02.401511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:02.401519 (+ 8us) server_negotiation.cc:407] Connection header received
0619 17:01:02.401551 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:02.401552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:02.401565 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:02.401656 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:01:02.401788 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.403014 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.404403 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.405138 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.405208 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:02.405262 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:02.405657 (+ 395us) server_negotiation.cc:299] Negotiation successful
0619 17:01:02.405704 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":93,"threads_started":1}
I20260619 17:01:02.406003 3670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:02.401252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45034 (local address 127.20.231.130:42577)
0619 17:01:02.401792 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:02.401797 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:02.401806 (+ 9us) server_negotiation.cc:407] Connection header received
0619 17:01:02.401847 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:02.401849 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:02.401863 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:02.402064 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:01:02.402140 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.403208 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.404590 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.405349 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.405800 (+ 451us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:02.405866 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:02.405892 (+ 26us) server_negotiation.cc:299] Negotiation successful
0619 17:01:02.405932 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":95,"threads_started":1}
I20260619 17:01:02.406440 3595 tablet_service.cc:1467] Processing CreateTablet for tablet de40b7a929314c468cc3617774433870 (DEFAULT_TABLE table=test-table [id=d0713b9d199a441aa048b9ad78355b59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 17:01:02.406575 3464 tablet_service.cc:1467] Processing CreateTablet for tablet de40b7a929314c468cc3617774433870 (DEFAULT_TABLE table=test-table [id=d0713b9d199a441aa048b9ad78355b59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 17:01:02.406808 3595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de40b7a929314c468cc3617774433870. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:01:02.406819 3464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de40b7a929314c468cc3617774433870. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:01:02.409638 3672 tablet_bootstrap.cc:492] T de40b7a929314c468cc3617774433870 P 14686394078f408d8f2970b5a0218323: Bootstrap starting.
I20260619 17:01:02.410244 3672 tablet_bootstrap.cc:654] T de40b7a929314c468cc3617774433870 P 14686394078f408d8f2970b5a0218323: Neither blocks nor log segments found. Creating new log.
I20260619 17:01:02.410899 3672 log.cc:824] T de40b7a929314c468cc3617774433870 P 14686394078f408d8f2970b5a0218323: Log is configured to *not* fsync() on all Append() calls
I20260619 17:01:02.411026 3673 tablet_bootstrap.cc:492] T de40b7a929314c468cc3617774433870 P 86ed8d3ae791430d8acf249777ed79e1: Bootstrap starting.
I20260619 17:01:02.411620 3673 tablet_bootstrap.cc:654] T de40b7a929314c468cc3617774433870 P 86ed8d3ae791430d8acf249777ed79e1: Neither blocks nor log segments found. Creating new log.
I20260619 17:01:02.411741 3672 tablet_bootstrap.cc:492] T de40b7a929314c468cc3617774433870 P 14686394078f408d8f2970b5a0218323: No bootstrap required, opened a new log
I20260619 17:01:02.411813 3672 ts_tablet_manager.cc:1397] T de40b7a929314c468cc3617774433870 P 14686394078f408d8f2970b5a0218323: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 17:01:02.412137 3673 log.cc:824] T de40b7a929314c468cc3617774433870 P 86ed8d3ae791430d8acf249777ed79e1: Log is configured to *not* fsync() on all Append() calls
I20260619 17:01:02.412866 3673 tablet_bootstrap.cc:492] T de40b7a929314c468cc3617774433870 P 86ed8d3ae791430d8acf249777ed79e1: No bootstrap required, opened a new log
I20260619 17:01:02.412941 3673 ts_tablet_manager.cc:1397] T de40b7a929314c468cc3617774433870 P 86ed8d3ae791430d8acf249777ed79e1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 17:01:02.413455 3672 raft_consensus.cc:348] T de40b7a929314c468cc3617774433870 P 14686394078f408d8f2970b5a0218323 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86ed8d3ae791430d8acf249777ed79e1" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42577 } } peers { permanent_uuid: "14686394078f408d8f2970b5a0218323" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36657 } } peers { permanent_uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 38807 } }
I20260619 17:01:02.413640 3672 raft_consensus.cc:374] T de40b7a929314c468cc3617774433870 P 14686394078f408d8f2970b5a0218323 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:01:02.413687 3672 raft_consensus.cc:729] T de40b7a929314c468cc3617774433870 P 14686394078f408d8f2970b5a0218323 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14686394078f408d8f2970b5a0218323, State: Initialized, Role: FOLLOWER
I20260619 17:01:02.413807 3672 consensus_queue.cc:260] T de40b7a929314c468cc3617774433870 P 14686394078f408d8f2970b5a0218323 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86ed8d3ae791430d8acf249777ed79e1" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42577 } } peers { permanent_uuid: "14686394078f408d8f2970b5a0218323" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36657 } } peers { permanent_uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 38807 } }
I20260619 17:01:02.414100 3662 heartbeater.cc:503] Master 127.20.231.190:33699 was elected leader, sending a full tablet report...
I20260619 17:01:02.414108 3672 ts_tablet_manager.cc:1428] T de40b7a929314c468cc3617774433870 P 14686394078f408d8f2970b5a0218323: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 17:01:02.414681 3673 raft_consensus.cc:348] T de40b7a929314c468cc3617774433870 P 86ed8d3ae791430d8acf249777ed79e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86ed8d3ae791430d8acf249777ed79e1" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42577 } } peers { permanent_uuid: "14686394078f408d8f2970b5a0218323" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36657 } } peers { permanent_uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 38807 } }
I20260619 17:01:02.414891 3673 raft_consensus.cc:374] T de40b7a929314c468cc3617774433870 P 86ed8d3ae791430d8acf249777ed79e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:01:02.414943 3673 raft_consensus.cc:729] T de40b7a929314c468cc3617774433870 P 86ed8d3ae791430d8acf249777ed79e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86ed8d3ae791430d8acf249777ed79e1, State: Initialized, Role: FOLLOWER
I20260619 17:01:02.415062 3673 consensus_queue.cc:260] T de40b7a929314c468cc3617774433870 P 86ed8d3ae791430d8acf249777ed79e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86ed8d3ae791430d8acf249777ed79e1" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42577 } } peers { permanent_uuid: "14686394078f408d8f2970b5a0218323" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36657 } } peers { permanent_uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 38807 } }
I20260619 17:01:02.415336 3673 ts_tablet_manager.cc:1428] T de40b7a929314c468cc3617774433870 P 86ed8d3ae791430d8acf249777ed79e1: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260619 17:01:02.415364 3530 heartbeater.cc:503] Master 127.20.231.190:33699 was elected leader, sending a full tablet report...
W20260619 17:01:02.499351 3531 tablet.cc:2369] T de40b7a929314c468cc3617774433870 P 86ed8d3ae791430d8acf249777ed79e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 17:01:02.595688 3393 tablet.cc:2369] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 17:01:02.630000 3663 tablet.cc:2369] T de40b7a929314c468cc3617774433870 P 14686394078f408d8f2970b5a0218323: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 17:01:02.644795 3676 raft_consensus.cc:482] T de40b7a929314c468cc3617774433870 P 14686394078f408d8f2970b5a0218323 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 17:01:02.644989 3676 raft_consensus.cc:504] T de40b7a929314c468cc3617774433870 P 14686394078f408d8f2970b5a0218323 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86ed8d3ae791430d8acf249777ed79e1" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42577 } } peers { permanent_uuid: "14686394078f408d8f2970b5a0218323" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36657 } } peers { permanent_uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 38807 } }
I20260619 17:01:02.645432 3676 leader_election.cc:288] T de40b7a929314c468cc3617774433870 P 14686394078f408d8f2970b5a0218323 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 86ed8d3ae791430d8acf249777ed79e1 (127.20.231.130:42577), 82b7a7a6f8ce4fa798108c74c2b99ab4 (127.20.231.129:38807)
I20260619 17:01:02.649520 3670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:02.645766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:37087 (local address 127.20.231.130:42577)
0619 17:01:02.645828 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:02.645830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:02.645840 (+ 10us) server_negotiation.cc:407] Connection header received
0619 17:01:02.645873 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:02.645874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:02.645884 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:02.645996 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:01:02.646160 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.646984 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.648437 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.649087 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.649188 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:02.649209 (+ 21us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:02.649380 (+ 171us) server_negotiation.cc:299] Negotiation successful
0619 17:01:02.649439 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 17:01:02.649521 3679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:02.645760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:40167 (local address 127.20.231.129:38807)
0619 17:01:02.645889 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:02.645892 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:02.645926 (+ 34us) server_negotiation.cc:407] Connection header received
0619 17:01:02.646028 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:02.646029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:02.646040 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:02.646111 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:01:02.646265 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.647091 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.648437 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.649152 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.649237 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:02.649245 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:02.649380 (+ 135us) server_negotiation.cc:299] Negotiation successful
0619 17:01:02.649437 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":62,"threads_started":1}
I20260619 17:01:02.649549 3678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:02.645668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:38807 (local address 127.20.231.131:40167)
0619 17:01:02.645865 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:02.645882 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 17:01:02.645931 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:01:02.646135 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:01:02.646136 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:01:02.646138 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:01:02.646198 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:02.646200 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.647165 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.647166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:02.648311 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:02.648313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.649229 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.649229 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:02.649290 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:02.649295 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 17:01:02.649357 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 17:01:02.649443 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":26,"thread_start_us":114,"threads_started":1}
I20260619 17:01:02.649554 3664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:02.645668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:42577 (local address 127.20.231.131:37087)
0619 17:01:02.645726 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:02.645738 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 17:01:02.645783 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:01:02.645992 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:01:02.645993 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:01:02.645995 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:01:02.646070 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:02.646071 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.647072 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.647073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:02.648311 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:02.648313 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.649182 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.649183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:02.649290 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:02.649295 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 17:01:02.649357 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 17:01:02.649444 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":19}
I20260619 17:01:02.650027 3346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de40b7a929314c468cc3617774433870" candidate_uuid: "14686394078f408d8f2970b5a0218323" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4" is_pre_election: true
I20260619 17:01:02.650135 3484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de40b7a929314c468cc3617774433870" candidate_uuid: "14686394078f408d8f2970b5a0218323" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86ed8d3ae791430d8acf249777ed79e1" is_pre_election: true
I20260619 17:01:02.650228 3346 raft_consensus.cc:2454] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14686394078f408d8f2970b5a0218323 in term 0.
I20260619 17:01:02.650322 3484 raft_consensus.cc:2454] T de40b7a929314c468cc3617774433870 P 86ed8d3ae791430d8acf249777ed79e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 14686394078f408d8f2970b5a0218323 in term 0.
I20260619 17:01:02.650486 3552 leader_election.cc:302] T de40b7a929314c468cc3617774433870 P 14686394078f408d8f2970b5a0218323 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14686394078f408d8f2970b5a0218323, 82b7a7a6f8ce4fa798108c74c2b99ab4; no voters:
I20260619 17:01:02.650673 3676 raft_consensus.cc:2781] T de40b7a929314c468cc3617774433870 P 14686394078f408d8f2970b5a0218323 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 17:01:02.650878 3676 raft_consensus.cc:482] T de40b7a929314c468cc3617774433870 P 14686394078f408d8f2970b5a0218323 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 17:01:02.650926 3676 raft_consensus.cc:3037] T de40b7a929314c468cc3617774433870 P 14686394078f408d8f2970b5a0218323 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:01:02.651670 3676 raft_consensus.cc:504] T de40b7a929314c468cc3617774433870 P 14686394078f408d8f2970b5a0218323 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86ed8d3ae791430d8acf249777ed79e1" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42577 } } peers { permanent_uuid: "14686394078f408d8f2970b5a0218323" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36657 } } peers { permanent_uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 38807 } }
I20260619 17:01:02.651844 3676 leader_election.cc:288] T de40b7a929314c468cc3617774433870 P 14686394078f408d8f2970b5a0218323 [CANDIDATE]: Term 1 election: Requested vote from peers 86ed8d3ae791430d8acf249777ed79e1 (127.20.231.130:42577), 82b7a7a6f8ce4fa798108c74c2b99ab4 (127.20.231.129:38807)
I20260619 17:01:02.652011 3346 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de40b7a929314c468cc3617774433870" candidate_uuid: "14686394078f408d8f2970b5a0218323" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4"
I20260619 17:01:02.652019 3484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de40b7a929314c468cc3617774433870" candidate_uuid: "14686394078f408d8f2970b5a0218323" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "86ed8d3ae791430d8acf249777ed79e1"
I20260619 17:01:02.652096 3346 raft_consensus.cc:3037] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:01:02.652122 3484 raft_consensus.cc:3037] T de40b7a929314c468cc3617774433870 P 86ed8d3ae791430d8acf249777ed79e1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:01:02.652799 3346 raft_consensus.cc:2454] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14686394078f408d8f2970b5a0218323 in term 1.
I20260619 17:01:02.652927 3484 raft_consensus.cc:2454] T de40b7a929314c468cc3617774433870 P 86ed8d3ae791430d8acf249777ed79e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 14686394078f408d8f2970b5a0218323 in term 1.
I20260619 17:01:02.652968 3552 leader_election.cc:302] T de40b7a929314c468cc3617774433870 P 14686394078f408d8f2970b5a0218323 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 14686394078f408d8f2970b5a0218323, 82b7a7a6f8ce4fa798108c74c2b99ab4; no voters:
I20260619 17:01:02.653043 3676 raft_consensus.cc:2781] T de40b7a929314c468cc3617774433870 P 14686394078f408d8f2970b5a0218323 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:01:02.653214 3676 raft_consensus.cc:686] T de40b7a929314c468cc3617774433870 P 14686394078f408d8f2970b5a0218323 [term 1 LEADER]: Becoming Leader. State: Replica: 14686394078f408d8f2970b5a0218323, State: Running, Role: LEADER
I20260619 17:01:02.653317 3676 consensus_queue.cc:237] T de40b7a929314c468cc3617774433870 P 14686394078f408d8f2970b5a0218323 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86ed8d3ae791430d8acf249777ed79e1" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42577 } } peers { permanent_uuid: "14686394078f408d8f2970b5a0218323" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36657 } } peers { permanent_uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 38807 } }
I20260619 17:01:02.654233 3203 catalog_manager.cc:5495] T de40b7a929314c468cc3617774433870 P 14686394078f408d8f2970b5a0218323 reported cstate change: term changed from 0 to 1, leader changed from <none> to 14686394078f408d8f2970b5a0218323 (127.20.231.131). New cstate: current_term: 1 leader_uuid: "14686394078f408d8f2970b5a0218323" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86ed8d3ae791430d8acf249777ed79e1" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "14686394078f408d8f2970b5a0218323" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36657 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 38807 } health_report { overall_health: UNKNOWN } } }
I20260619 17:01:02.690734 3669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:02.687960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45080 (local address 127.20.231.131:36657)
0619 17:01:02.688036 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:02.688038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:02.688052 (+ 14us) server_negotiation.cc:407] Connection header received
0619 17:01:02.688101 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:02.688101 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:02.688124 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:02.688222 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 17:01:02.688366 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.688897 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.689763 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:02.689998 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:02.690104 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:02.690286 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 17:01:02.690407 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 17:01:02.690514 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:02.690578 (+ 64us) server_negotiation.cc:299] Negotiation successful
0619 17:01:02.690600 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 17:01:02.692694 3484 raft_consensus.cc:1261] T de40b7a929314c468cc3617774433870 P 86ed8d3ae791430d8acf249777ed79e1 [term 1 FOLLOWER]: Refusing update from remote peer 14686394078f408d8f2970b5a0218323: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 17:01:02.692672 3346 raft_consensus.cc:1261] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [term 1 FOLLOWER]: Refusing update from remote peer 14686394078f408d8f2970b5a0218323: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 17:01:02.693153 3680 consensus_queue.cc:1035] T de40b7a929314c468cc3617774433870 P 14686394078f408d8f2970b5a0218323 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 38807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 17:01:02.693300 3676 consensus_queue.cc:1035] T de40b7a929314c468cc3617774433870 P 14686394078f408d8f2970b5a0218323 [LEADER]: Connected to new peer: Peer: permanent_uuid: "86ed8d3ae791430d8acf249777ed79e1" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 17:01:02.695537 3683 mvcc.cc:205] Tried to move back new op lower bound from 7298615143186034688 to 7298615143028842496. Current Snapshot: MvccSnapshot[applied={T|T < 7298615143186034688}]
I20260619 17:01:02.695638 3684 mvcc.cc:205] Tried to move back new op lower bound from 7298615143186034688 to 7298615143028842496. Current Snapshot: MvccSnapshot[applied={T|T < 7298615143186034688}]
I20260619 17:01:02.698091 3202 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:43114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 17:01:02.698210 3202 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:43114:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 17:01:02.699019 3202 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087: Sending DeleteTablet for 3 replicas of tablet de40b7a929314c468cc3617774433870
I20260619 17:01:02.699380 3464 tablet_service.cc:1514] Processing DeleteTablet for tablet de40b7a929314c468cc3617774433870 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:01:02 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:45034
I20260619 17:01:02.699434 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 3241
I20260619 17:01:02.699604 3326 tablet_service.cc:1514] Processing DeleteTablet for tablet de40b7a929314c468cc3617774433870 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:01:02 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:41140
I20260619 17:01:02.700377 3693 tablet_replica.cc:330] T de40b7a929314c468cc3617774433870 P 86ed8d3ae791430d8acf249777ed79e1: stopping tablet replica
I20260619 17:01:02.700353 3595 tablet_service.cc:1514] Processing DeleteTablet for tablet de40b7a929314c468cc3617774433870 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:01:02 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:45064
I20260619 17:01:02.700484 3693 raft_consensus.cc:2229] T de40b7a929314c468cc3617774433870 P 86ed8d3ae791430d8acf249777ed79e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 17:01:02.700593 3693 raft_consensus.cc:2258] T de40b7a929314c468cc3617774433870 P 86ed8d3ae791430d8acf249777ed79e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 17:01:02.700766 3694 tablet_replica.cc:330] T de40b7a929314c468cc3617774433870 P 14686394078f408d8f2970b5a0218323: stopping tablet replica
I20260619 17:01:02.700857 3694 raft_consensus.cc:2229] T de40b7a929314c468cc3617774433870 P 14686394078f408d8f2970b5a0218323 [term 1 LEADER]: Raft consensus shutting down.
I20260619 17:01:02.700956 3694 raft_consensus.cc:2258] T de40b7a929314c468cc3617774433870 P 14686394078f408d8f2970b5a0218323 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 17:01:02.701336 3693 ts_tablet_manager.cc:1905] T de40b7a929314c468cc3617774433870 P 86ed8d3ae791430d8acf249777ed79e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 17:01:02.701359 3694 ts_tablet_manager.cc:1905] T de40b7a929314c468cc3617774433870 P 14686394078f408d8f2970b5a0218323: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 17:01:02.703346 3693 ts_tablet_manager.cc:1918] T de40b7a929314c468cc3617774433870 P 86ed8d3ae791430d8acf249777ed79e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 17:01:02.703346 3694 ts_tablet_manager.cc:1918] T de40b7a929314c468cc3617774433870 P 14686394078f408d8f2970b5a0218323: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 17:01:02.703428 3693 log.cc:1196] T de40b7a929314c468cc3617774433870 P 86ed8d3ae791430d8acf249777ed79e1: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/wal/wals/de40b7a929314c468cc3617774433870
I20260619 17:01:02.703532 3694 log.cc:1196] T de40b7a929314c468cc3617774433870 P 14686394078f408d8f2970b5a0218323: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/wal/wals/de40b7a929314c468cc3617774433870
I20260619 17:01:02.703738 3693 ts_tablet_manager.cc:1939] T de40b7a929314c468cc3617774433870 P 86ed8d3ae791430d8acf249777ed79e1: Deleting consensus metadata
I20260619 17:01:02.703923 3694 ts_tablet_manager.cc:1939] T de40b7a929314c468cc3617774433870 P 14686394078f408d8f2970b5a0218323: Deleting consensus metadata
I20260619 17:01:02.704229 3188 catalog_manager.cc:4841] TS 86ed8d3ae791430d8acf249777ed79e1 (127.20.231.130:42577): tablet de40b7a929314c468cc3617774433870 (table test-table [id=d0713b9d199a441aa048b9ad78355b59]) successfully deleted
I20260619 17:01:02.704396 3190 catalog_manager.cc:4841] TS 14686394078f408d8f2970b5a0218323 (127.20.231.131:36657): tablet de40b7a929314c468cc3617774433870 (table test-table [id=d0713b9d199a441aa048b9ad78355b59]) successfully deleted
W20260619 17:01:02.707551 3190 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.231.129:38807 (error 108)
I20260619 17:01:02.707990 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 3398
I20260619 17:01:02.708070 3399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:02.707898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:38807 (local address 127.0.0.1:41152)
0619 17:01:02.707954 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:02.708024 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.129:38807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260619 17:01:02.711385 3190 catalog_manager.cc:4584] TS 82b7a7a6f8ce4fa798108c74c2b99ab4 (127.20.231.129:38807): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet de40b7a929314c468cc3617774433870: Network error: Client connection negotiation failed: client connection to 127.20.231.129:38807: connect: Connection refused (error 111)
I20260619 17:01:02.714696 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 3533
I20260619 17:01:02.719470 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 3170
I20260619 17:01:02.725152 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:33699
--webserver_interface=127.20.231.190
--webserver_port=34589
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:37209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:33699
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:01:02.806391 3696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:01:02.806612 3696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:01:02.806636 3696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:01:02.807804 3696 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:01:02.807844 3696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:01:02.807859 3696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:01:02.807873 3696 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:01:02.807883 3696 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:01:02.809289 3696 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:37209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:33699
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:33699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=34589
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:01:02.809556 3696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:01:02.809770 3696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:01:02.814793 3704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:01:02.814787 3702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:01:02.814787 3701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:01:02.814805 3696 server_base.cc:1034] running on GCE node
I20260619 17:01:02.815351 3696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:01:02.815598 3696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:01:02.816735 3696 hybrid_clock.cc:648] HybridClock initialized: now 1781888462816762 us; error 70 us; skew 500 ppm
Jun 19 17:01:02 dist-test-slave-vnbj krb5kdc[2623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888462, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:01:02.819298 3696 init.cc:377] Logged in from keytab as kudu/127.20.231.190@KRBTEST.COM (short username kudu)
I20260619 17:01:02.819777 3696 webserver.cc:460] Webserver started at http://127.20.231.190:34589/ using document root <none> and password file <none>
I20260619 17:01:02.820022 3696 fs_manager.cc:359] Metadata directory not provided
I20260619 17:01:02.820092 3696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:01:02.821451 3696 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 17:01:02.822175 3711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:02.822371 3696 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:01:02.822475 3696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "7b6203e4ca6b4fb7b12e8804e23fc087"
format_stamp: "Formatted at 2026-06-19 17:01:01 on dist-test-slave-vnbj"
server_key: "68f94e1b11156675e8acd0acbc1edca7"
server_key_iv: "905786f24e1770660bd75d86247322ea"
server_key_version: "encryptionkey@0"
I20260619 17:01:02.822845 3696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:01:02.844524 3696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:01:02.846627 3696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:01:02.846792 3696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:01:02.851208 3696 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:33699
I20260619 17:01:02.851271 3763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:33699 every 8 connection(s)
I20260619 17:01:02.851656 3696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 17:01:02.853433 3764 sys_catalog.cc:263] Verifying existing consensus state
I20260619 17:01:02.854079 3764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087: Bootstrap starting.
I20260619 17:01:02.860519 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 3696
I20260619 17:01:02.860901 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.129:38807
--local_ip_for_outbound_sockets=127.20.231.129
--tserver_master_addrs=127.20.231.190:33699
--webserver_port=44057
--webserver_interface=127.20.231.129
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:37209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 17:01:02.863442 3764 log.cc:824] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087: Log is configured to *not* fsync() on all Append() calls
I20260619 17:01:02.865842 3764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 17:01:02.866194 3764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087: Bootstrap complete.
I20260619 17:01:02.868494 3764 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6203e4ca6b4fb7b12e8804e23fc087" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 33699 } }
I20260619 17:01:02.868718 3764 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b6203e4ca6b4fb7b12e8804e23fc087, State: Initialized, Role: FOLLOWER
I20260619 17:01:02.868891 3764 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6203e4ca6b4fb7b12e8804e23fc087" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 33699 } }
I20260619 17:01:02.868963 3764 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 17:01:02.869010 3764 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 17:01:02.869060 3764 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [term 2 FOLLOWER]: Advancing to term 3
I20260619 17:01:02.870131 3764 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6203e4ca6b4fb7b12e8804e23fc087" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 33699 } }
I20260619 17:01:02.870225 3764 leader_election.cc:302] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b6203e4ca6b4fb7b12e8804e23fc087; no voters:
I20260619 17:01:02.870460 3764 leader_election.cc:288] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 17:01:02.870543 3768 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 17:01:02.870748 3764 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:01:02.870746 3768 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [term 3 LEADER]: Becoming Leader. State: Replica: 7b6203e4ca6b4fb7b12e8804e23fc087, State: Running, Role: LEADER
I20260619 17:01:02.870896 3768 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6203e4ca6b4fb7b12e8804e23fc087" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 33699 } }
I20260619 17:01:02.871178 3769 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "7b6203e4ca6b4fb7b12e8804e23fc087" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6203e4ca6b4fb7b12e8804e23fc087" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 33699 } } }
I20260619 17:01:02.871264 3769 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [sys.catalog]: This master's current role is: LEADER
I20260619 17:01:02.871181 3770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b6203e4ca6b4fb7b12e8804e23fc087. Latest consensus state: current_term: 3 leader_uuid: "7b6203e4ca6b4fb7b12e8804e23fc087" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b6203e4ca6b4fb7b12e8804e23fc087" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 33699 } } }
I20260619 17:01:02.871439 3770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b6203e4ca6b4fb7b12e8804e23fc087 [sys.catalog]: This master's current role is: LEADER
I20260619 17:01:02.871786 3774 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 17:01:02.872574 3774 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 17:01:02.872674 3774 catalog_manager.cc:1259] Loaded cluster ID: 4bc0593d767e445fa2edfb11df956b67
I20260619 17:01:02.872706 3774 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260619 17:01:02.872949 3774 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fae88f259ad google::LogMessage::Fail() at ??:0
@ 0x7fae88f2888a google::LogMessage::SendToLog() at ??:0
@ 0x7fae88f254de google::LogMessage::Flush() at ??:0
@ 0x7fae88f26a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fae90e4a0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fae90e538b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fae8adf30d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fae8ade8954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fae8bc4c6db start_thread at ??:0
@ 0x7fae87a2361f clone at ??:0
W20260619 17:01:02.994765 3766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:01:02.995129 3766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:01:02.995172 3766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:01:02.997169 3766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:01:02.997237 3766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:01:02.997283 3766 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.129
I20260619 17:01:02.999667 3766 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:37209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.129:38807
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.231.129
--webserver_port=44057
--tserver_master_addrs=127.20.231.190:33699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.129
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:01:02.999969 3766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:01:03.000273 3766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:01:03.006177 3803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:01:03.006227 3800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:01:03.006206 3801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:01:03.006596 3766 server_base.cc:1034] running on GCE node
I20260619 17:01:03.006839 3766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:01:03.007153 3766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:01:03.008319 3766 hybrid_clock.cc:648] HybridClock initialized: now 1781888463008297 us; error 41 us; skew 500 ppm
Jun 19 17:01:03 dist-test-slave-vnbj krb5kdc[2623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888463, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:01:03.011191 3766 init.cc:377] Logged in from keytab as kudu/127.20.231.129@KRBTEST.COM (short username kudu)
I20260619 17:01:03.011739 3766 webserver.cc:460] Webserver started at http://127.20.231.129:44057/ using document root <none> and password file <none>
I20260619 17:01:03.011986 3766 fs_manager.cc:359] Metadata directory not provided
I20260619 17:01:03.012063 3766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:01:03.013597 3766 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:01:03.014384 3810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:03.014554 3766 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:01:03.014655 3766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/wal
uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4"
format_stamp: "Formatted at 2026-06-19 17:01:01 on dist-test-slave-vnbj"
server_key: "0342c76ec53e01a9a502777e24968d93"
server_key_iv: "c3fb1de100370d6adf7f8d2a2fc0a550"
server_key_version: "encryptionkey@0"
I20260619 17:01:03.015085 3766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:01:03.032487 3766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:01:03.035133 3766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:01:03.035259 3766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:01:03.035513 3766 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:01:03.036001 3817 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 17:01:03.038625 3766 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 17:01:03.038697 3766 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.003s user 0.000s sys 0.000s
I20260619 17:01:03.038743 3766 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 17:01:03.039453 3766 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 17:01:03.039515 3766 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 17:01:03.039582 3817 tablet_bootstrap.cc:492] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4: Bootstrap starting.
I20260619 17:01:03.047013 3766 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.129:38807
I20260619 17:01:03.047113 3924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.129:38807 every 8 connection(s)
I20260619 17:01:03.047456 3766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
I20260619 17:01:03.051244 3817 log.cc:824] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4: Log is configured to *not* fsync() on all Append() calls
I20260619 17:01:03.054225 3817 tablet_bootstrap.cc:492] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 17:01:03.054562 3817 tablet_bootstrap.cc:492] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4: Bootstrap complete.
I20260619 17:01:03.055863 3817 ts_tablet_manager.cc:1397] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.008s
I20260619 17:01:03.057391 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 3766
I20260619 17:01:03.057647 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.130:42577
--local_ip_for_outbound_sockets=127.20.231.130
--tserver_master_addrs=127.20.231.190:33699
--webserver_port=41793
--webserver_interface=127.20.231.130
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:37209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 17:01:03.057471 3817 raft_consensus.cc:348] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86ed8d3ae791430d8acf249777ed79e1" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42577 } } peers { permanent_uuid: "14686394078f408d8f2970b5a0218323" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36657 } } peers { permanent_uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 38807 } }
I20260619 17:01:03.057796 3817 raft_consensus.cc:729] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82b7a7a6f8ce4fa798108c74c2b99ab4, State: Initialized, Role: FOLLOWER
I20260619 17:01:03.057976 3817 consensus_queue.cc:260] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86ed8d3ae791430d8acf249777ed79e1" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42577 } } peers { permanent_uuid: "14686394078f408d8f2970b5a0218323" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36657 } } peers { permanent_uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 38807 } }
I20260619 17:01:03.058281 3817 ts_tablet_manager.cc:1428] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4: Time spent starting tablet: real 0.002s user 0.001s sys 0.003s
W20260619 17:01:03.145733 3927 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 17:01:03.048137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.129:48635)
0619 17:01:03.048318 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:03.048365 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 17:01:03.048824 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:01:03.145507 (+ 96683us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":98,"threads_started":1}
W20260619 17:01:03.146066 3925 heartbeater.cc:644] Failed to heartbeat to 127.20.231.190:33699 (0 consecutive failures): Network error: Failed to ping master at 127.20.231.190:33699: Client connection negotiation failed: client connection to 127.20.231.190:33699: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260619 17:01:03.145915 3930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:01:03.146203 3930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:01:03.146267 3930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260619 17:01:03.146520 3927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:03.146395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.129:47331)
0619 17:01:03.146448 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:03.146496 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 17:01:03.146930 3927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:03.146795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.129:51765)
0619 17:01:03.146880 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:03.146908 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 17:01:03.147061 3925 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 17:01:03.147868 3930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:01:03.147913 3930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:01:03.147944 3930 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.130
I20260619 17:01:03.149564 3930 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:37209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.130:42577
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.231.130
--webserver_port=41793
--tserver_master_addrs=127.20.231.190:33699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.130
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:01:03.149784 3930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:01:03.150070 3930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:01:03.155467 3936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:01:03.155632 3930 server_base.cc:1034] running on GCE node
W20260619 17:01:03.155472 3935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:01:03.155512 3938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:01:03.156004 3930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:01:03.156229 3930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:01:03.157375 3930 hybrid_clock.cc:648] HybridClock initialized: now 1781888463157366 us; error 32 us; skew 500 ppm
Jun 19 17:01:03 dist-test-slave-vnbj krb5kdc[2623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888463, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:01:03.160259 3930 init.cc:377] Logged in from keytab as kudu/127.20.231.130@KRBTEST.COM (short username kudu)
I20260619 17:01:03.160671 3930 webserver.cc:460] Webserver started at http://127.20.231.130:41793/ using document root <none> and password file <none>
I20260619 17:01:03.160887 3930 fs_manager.cc:359] Metadata directory not provided
I20260619 17:01:03.160931 3930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:01:03.162249 3930 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 17:01:03.162953 3945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:03.163139 3930 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 17:01:03.163242 3930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/wal
uuid: "86ed8d3ae791430d8acf249777ed79e1"
format_stamp: "Formatted at 2026-06-19 17:01:01 on dist-test-slave-vnbj"
server_key: "363d46d09da75bca68799764d9f19455"
server_key_iv: "ff210e9fefa1a5a5779cb09c9bd7eee9"
server_key_version: "encryptionkey@0"
I20260619 17:01:03.163628 3930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:01:03.174470 3930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:01:03.176908 3930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:01:03.177083 3930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:01:03.177342 3930 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:01:03.177660 3930 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:01:03.177723 3930 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:03.177785 3930 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:01:03.177814 3930 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:03.183759 3930 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.130:42577
I20260619 17:01:03.183809 4058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.130:42577 every 8 connection(s)
I20260619 17:01:03.184144 3930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
I20260619 17:01:03.185123 4061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:03.184743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.130:38457)
0619 17:01:03.184903 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:03.184984 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":66,"threads_started":1}
W20260619 17:01:03.185489 4059 heartbeater.cc:644] Failed to heartbeat to 127.20.231.190:33699 (0 consecutive failures): Network error: Failed to ping master at 127.20.231.190:33699: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
I20260619 17:01:03.185850 4061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:03.185690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.130:44073)
0619 17:01:03.185807 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:03.185833 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260619 17:01:03.186148 4061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:03.186075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.130:57113)
0619 17:01:03.186113 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:03.186134 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260619 17:01:03.186267 4059 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 17:01:03.193871 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 3930
I20260619 17:01:03.194067 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.131:36657
--local_ip_for_outbound_sockets=127.20.231.131
--tserver_master_addrs=127.20.231.190:33699
--webserver_port=32951
--webserver_interface=127.20.231.131
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:37209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:01:03.279206 4062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:01:03.279443 4062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:01:03.279485 4062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:01:03.280763 4062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:01:03.280810 4062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:01:03.280840 4062 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.131
I20260619 17:01:03.282423 4062 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:37209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.131:36657
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.231.131
--webserver_port=32951
--tserver_master_addrs=127.20.231.190:33699
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.131
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:01:03.282630 4062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:01:03.282878 4062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:01:03.288751 4070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:01:03.288756 4067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:01:03.288753 4068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:01:03.289155 4062 server_base.cc:1034] running on GCE node
I20260619 17:01:03.289394 4062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:01:03.289692 4062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:01:03.290930 4062 hybrid_clock.cc:648] HybridClock initialized: now 1781888463290913 us; error 49 us; skew 500 ppm
Jun 19 17:01:03 dist-test-slave-vnbj krb5kdc[2623](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888463, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:01:03.293671 4062 init.cc:377] Logged in from keytab as kudu/127.20.231.131@KRBTEST.COM (short username kudu)
I20260619 17:01:03.294247 4062 webserver.cc:460] Webserver started at http://127.20.231.131:32951/ using document root <none> and password file <none>
I20260619 17:01:03.294503 4062 fs_manager.cc:359] Metadata directory not provided
I20260619 17:01:03.294577 4062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:01:03.296072 4062 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:01:03.296829 4077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:03.297055 4062 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:01:03.297165 4062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/wal
uuid: "14686394078f408d8f2970b5a0218323"
format_stamp: "Formatted at 2026-06-19 17:01:01 on dist-test-slave-vnbj"
server_key: "8ae8083cb7a9b1783a580038e77b6d49"
server_key_iv: "78a34c3b8a3f7c1b78621f8d6f1f02a1"
server_key_version: "encryptionkey@0"
I20260619 17:01:03.297534 4062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:01:03.316702 4062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:01:03.318885 4062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:01:03.319080 4062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:01:03.319360 4062 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:01:03.319676 4062 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:01:03.319732 4062 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:03.319767 4062 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:01:03.319813 4062 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:03.325805 4062 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.131:36657
I20260619 17:01:03.325850 4190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.131:36657 every 8 connection(s)
I20260619 17:01:03.326210 4062 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
I20260619 17:01:03.327397 4193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:03.326986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.131:39863)
0619 17:01:03.327165 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:03.327245 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":91,"threads_started":1}
W20260619 17:01:03.327814 4191 heartbeater.cc:644] Failed to heartbeat to 127.20.231.190:33699 (0 consecutive failures): Network error: Failed to ping master at 127.20.231.190:33699: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
I20260619 17:01:03.328171 4193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:03.328072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.131:59095)
0619 17:01:03.328124 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:03.328152 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 17:01:03.328534 4193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:03.328414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.131:56557)
0619 17:01:03.328479 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:03.328512 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260619 17:01:03.328685 4191 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 17:01:03.329368 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 4062
I20260619 17:01:04.147773 4195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:04.147489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.129:60919)
0619 17:01:04.147664 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:04.147710 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":88,"threads_started":1}
I20260619 17:01:04.186792 4196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:04.186585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.130:48755)
0619 17:01:04.186709 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:04.186742 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":71,"threads_started":1}
I20260619 17:01:04.279609 4197 raft_consensus.cc:482] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 17:01:04.279728 4197 raft_consensus.cc:504] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86ed8d3ae791430d8acf249777ed79e1" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42577 } } peers { permanent_uuid: "14686394078f408d8f2970b5a0218323" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36657 } } peers { permanent_uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 38807 } }
I20260619 17:01:04.280110 4197 leader_election.cc:288] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 86ed8d3ae791430d8acf249777ed79e1 (127.20.231.130:42577), 14686394078f408d8f2970b5a0218323 (127.20.231.131:36657)
Jun 19 17:01:04 dist-test-slave-vnbj krb5kdc[2623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888463, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for kudu/127.20.231.131@KRBTEST.COM
Jun 19 17:01:04 dist-test-slave-vnbj krb5kdc[2623](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888463, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for kudu/127.20.231.130@KRBTEST.COM
I20260619 17:01:04.291416 4198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:04.280362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:36657 (local address 127.20.231.129:40521)
0619 17:01:04.280513 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:04.280527 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 17:01:04.280600 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:01:04.281640 (+ 1040us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:01:04.281641 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:01:04.281970 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:01:04.282190 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:04.282196 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:04.283570 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:04.283571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:04.284318 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:04.284320 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:04.285001 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:04.285002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:04.285078 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:04.285759 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:01:04.285763 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:01:04.287394 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:01:04.289585 (+ 2191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:01:04.289594 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:01:04.289596 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:01:04.289844 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:01:04.290178 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:01:04.290179 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:01:04.290179 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:01:04.290314 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:01:04.290831 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:01:04.290837 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:01:04.290996 (+ 159us) client_negotiation.cc:769] Sending connection context
0619 17:01:04.291108 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 17:01:04.291280 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":40,"thread_start_us":73,"threads_started":1}
I20260619 17:01:04.291464 4199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:04.280497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:40521 (local address 127.20.231.131:36657)
0619 17:01:04.280677 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:04.280691 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:04.280722 (+ 31us) server_negotiation.cc:407] Connection header received
0619 17:01:04.281241 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:04.281244 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:04.281298 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:04.281534 (+ 236us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:01:04.282318 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:04.283463 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:04.284411 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:04.284905 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:04.285032 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:04.287522 (+ 2490us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:01:04.287526 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:01:04.287526 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:01:04.287537 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:01:04.289470 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:04.289957 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:04.289961 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:04.289961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:04.290104 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:04.290385 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:04.290386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:04.290387 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:04.290750 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:01:04.290841 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:04.291185 (+ 344us) server_negotiation.cc:299] Negotiation successful
0619 17:01:04.291350 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":59,"threads_started":1}
I20260619 17:01:04.291558 4200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:04.280543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:51465 (local address 127.20.231.130:42577)
0619 17:01:04.280722 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:04.280731 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:04.280758 (+ 27us) server_negotiation.cc:407] Connection header received
0619 17:01:04.281188 (+ 430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:04.281190 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:04.281232 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:04.281350 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:01:04.282344 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:04.283281 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:04.284461 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:04.284849 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:04.284933 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:04.288148 (+ 3215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:01:04.288151 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:01:04.288152 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:01:04.288163 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:01:04.289944 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:04.290332 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:04.290335 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:04.290335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:04.290447 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:04.290663 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:04.290663 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:04.290664 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:04.290962 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:01:04.291045 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:04.291284 (+ 239us) server_negotiation.cc:299] Negotiation successful
0619 17:01:04.291442 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":81,"threads_started":1}
I20260619 17:01:04.291952 4195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:04.280362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:42577 (local address 127.20.231.129:51465)
0619 17:01:04.280432 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:04.280446 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 17:01:04.280516 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:01:04.281468 (+ 952us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:01:04.281471 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:01:04.281987 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:01:04.282191 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:04.282197 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:04.283387 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:04.283388 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:04.284319 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:04.284322 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:04.284947 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:04.284949 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:04.285082 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:04.285732 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:01:04.285744 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:01:04.288004 (+ 2260us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:01:04.290015 (+ 2011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:01:04.290016 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:01:04.290016 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:01:04.290244 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:01:04.290526 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:01:04.290527 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:01:04.290527 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:01:04.290593 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:01:04.291057 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:01:04.291058 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:01:04.291123 (+ 65us) client_negotiation.cc:769] Sending connection context
0619 17:01:04.291209 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 17:01:04.291350 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":75}
W20260619 17:01:04.292404 3814 leader_election.cc:341] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 14686394078f408d8f2970b5a0218323 (127.20.231.131:36657): Not found: Tablet not found: de40b7a929314c468cc3617774433870
W20260619 17:01:04.292655 3812 leader_election.cc:341] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 86ed8d3ae791430d8acf249777ed79e1 (127.20.231.130:42577): Not found: Tablet not found: de40b7a929314c468cc3617774433870
I20260619 17:01:04.292713 3812 leader_election.cc:302] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 82b7a7a6f8ce4fa798108c74c2b99ab4; no voters: 14686394078f408d8f2970b5a0218323, 86ed8d3ae791430d8acf249777ed79e1
I20260619 17:01:04.292874 4197 raft_consensus.cc:2726] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 17:01:04.329316 4201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:04.329107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.131:60445)
0619 17:01:04.329231 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:04.329277 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":69,"threads_started":1}
I20260619 17:01:05.148667 4202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:05.148401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.129:48379)
0619 17:01:05.148564 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:05.148614 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":65,"threads_started":1}
I20260619 17:01:05.187543 4203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:05.187314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.130:60641)
0619 17:01:05.187457 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:05.187503 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":80,"threads_started":1}
I20260619 17:01:05.330178 4204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:05.329917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.131:60485)
0619 17:01:05.330086 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:05.330133 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":67,"threads_started":1}
I20260619 17:01:05.953399 4205 raft_consensus.cc:482] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 17:01:05.953495 4205 raft_consensus.cc:504] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86ed8d3ae791430d8acf249777ed79e1" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42577 } } peers { permanent_uuid: "14686394078f408d8f2970b5a0218323" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36657 } } peers { permanent_uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 38807 } }
I20260619 17:01:05.953670 4205 leader_election.cc:288] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 86ed8d3ae791430d8acf249777ed79e1 (127.20.231.130:42577), 14686394078f408d8f2970b5a0218323 (127.20.231.131:36657)
W20260619 17:01:05.954077 3814 leader_election.cc:341] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 14686394078f408d8f2970b5a0218323 (127.20.231.131:36657): Not found: Tablet not found: de40b7a929314c468cc3617774433870
W20260619 17:01:05.954171 3812 leader_election.cc:341] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 86ed8d3ae791430d8acf249777ed79e1 (127.20.231.130:42577): Not found: Tablet not found: de40b7a929314c468cc3617774433870
I20260619 17:01:05.954249 3812 leader_election.cc:302] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 82b7a7a6f8ce4fa798108c74c2b99ab4; no voters: 14686394078f408d8f2970b5a0218323, 86ed8d3ae791430d8acf249777ed79e1
I20260619 17:01:05.954339 4205 raft_consensus.cc:2726] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 17:01:06.149842 4206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:06.149526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.129:35385)
0619 17:01:06.149729 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:06.149789 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":118,"threads_started":1}
I20260619 17:01:06.188266 4207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:06.188045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.130:41021)
0619 17:01:06.188183 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:06.188222 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":65,"threads_started":1}
I20260619 17:01:06.331082 4208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:06.330774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.131:39219)
0619 17:01:06.330969 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:06.331017 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":119,"threads_started":1}
I20260619 17:01:07.150630 4209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:07.150359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.129:39521)
0619 17:01:07.150538 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:07.150582 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":85,"threads_started":1}
I20260619 17:01:07.189102 4210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:07.188844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.130:38395)
0619 17:01:07.189003 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:07.189051 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":71,"threads_started":1}
I20260619 17:01:07.331918 4211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:07.331670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.131:58267)
0619 17:01:07.331828 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:07.331872 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":59,"threads_started":1}
I20260619 17:01:07.909866 4212 raft_consensus.cc:482] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 17:01:07.909993 4212 raft_consensus.cc:504] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86ed8d3ae791430d8acf249777ed79e1" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42577 } } peers { permanent_uuid: "14686394078f408d8f2970b5a0218323" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36657 } } peers { permanent_uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 38807 } }
I20260619 17:01:07.910303 4212 leader_election.cc:288] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 86ed8d3ae791430d8acf249777ed79e1 (127.20.231.130:42577), 14686394078f408d8f2970b5a0218323 (127.20.231.131:36657)
W20260619 17:01:07.910612 3812 leader_election.cc:341] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 86ed8d3ae791430d8acf249777ed79e1 (127.20.231.130:42577): Not found: Tablet not found: de40b7a929314c468cc3617774433870
W20260619 17:01:07.910696 3814 leader_election.cc:341] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 14686394078f408d8f2970b5a0218323 (127.20.231.131:36657): Not found: Tablet not found: de40b7a929314c468cc3617774433870
I20260619 17:01:07.910761 3814 leader_election.cc:302] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 82b7a7a6f8ce4fa798108c74c2b99ab4; no voters: 14686394078f408d8f2970b5a0218323, 86ed8d3ae791430d8acf249777ed79e1
I20260619 17:01:07.910831 4212 raft_consensus.cc:2726] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 17:01:08.151441 4213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:08.151202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.129:35015)
0619 17:01:08.151348 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:08.151394 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":82,"threads_started":1}
I20260619 17:01:08.189875 4214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:08.189665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.130:60671)
0619 17:01:08.189789 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:08.189829 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":60,"threads_started":1}
I20260619 17:01:08.332644 4215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:08.332422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.131:49623)
0619 17:01:08.332556 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:08.332599 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":75,"threads_started":1}
I20260619 17:01:09.152343 4216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:09.152090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.129:58749)
0619 17:01:09.152248 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:09.152295 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":64,"threads_started":1}
I20260619 17:01:09.190665 4217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:09.190392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.130:42405)
0619 17:01:09.190566 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:09.190617 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":77,"threads_started":1}
I20260619 17:01:09.333307 4218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:09.333109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.131:37463)
0619 17:01:09.333224 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:09.333262 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":60,"threads_started":1}
I20260619 17:01:10.153152 4219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:10.152920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.129:57415)
0619 17:01:10.153057 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:10.153105 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":68,"threads_started":1}
I20260619 17:01:10.191470 4220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:10.191236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.130:48717)
0619 17:01:10.191364 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:10.191420 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":61,"threads_started":1}
I20260619 17:01:10.334075 4221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:10.333842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.131:40381)
0619 17:01:10.333981 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:10.334028 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":72,"threads_started":1}
I20260619 17:01:11.154006 4222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:11.153747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.129:40415)
0619 17:01:11.153914 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:11.153955 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":67,"threads_started":1}
I20260619 17:01:11.192405 4223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:11.192109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.130:60379)
0619 17:01:11.192302 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:11.192352 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":70,"threads_started":1}
I20260619 17:01:11.334738 4224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:11.334538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.131:43131)
0619 17:01:11.334657 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:11.334692 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":66,"threads_started":1}
I20260619 17:01:11.623776 4225 raft_consensus.cc:482] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 17:01:11.623873 4225 raft_consensus.cc:504] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86ed8d3ae791430d8acf249777ed79e1" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42577 } } peers { permanent_uuid: "14686394078f408d8f2970b5a0218323" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36657 } } peers { permanent_uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 38807 } }
I20260619 17:01:11.624045 4225 leader_election.cc:288] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 86ed8d3ae791430d8acf249777ed79e1 (127.20.231.130:42577), 14686394078f408d8f2970b5a0218323 (127.20.231.131:36657)
W20260619 17:01:11.624469 3812 leader_election.cc:341] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 86ed8d3ae791430d8acf249777ed79e1 (127.20.231.130:42577): Not found: Tablet not found: de40b7a929314c468cc3617774433870
W20260619 17:01:11.624538 3814 leader_election.cc:341] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 14686394078f408d8f2970b5a0218323 (127.20.231.131:36657): Not found: Tablet not found: de40b7a929314c468cc3617774433870
I20260619 17:01:11.624615 3814 leader_election.cc:302] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 82b7a7a6f8ce4fa798108c74c2b99ab4; no voters: 14686394078f408d8f2970b5a0218323, 86ed8d3ae791430d8acf249777ed79e1
I20260619 17:01:11.624711 4225 raft_consensus.cc:2726] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 17:01:12.154907 4226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:12.154614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.129:48939)
0619 17:01:12.154758 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:12.154803 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":79,"threads_started":1}
I20260619 17:01:12.193259 4227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:12.193025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.130:45359)
0619 17:01:12.193164 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:12.193212 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":61,"threads_started":1}
I20260619 17:01:12.335510 4228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:12.335242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.131:42125)
0619 17:01:12.335423 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:12.335462 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":75,"threads_started":1}
I20260619 17:01:13.155820 4229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:13.155482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.129:55319)
0619 17:01:13.155665 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:13.155723 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":70,"threads_started":1}
I20260619 17:01:13.194166 4230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:13.193932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.130:53191)
0619 17:01:13.194079 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:13.194120 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":74,"threads_started":1}
I20260619 17:01:13.336264 4231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:13.336028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.131:45521)
0619 17:01:13.336173 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:13.336215 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":75,"threads_started":1}
I20260619 17:01:14.156615 4232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:14.156409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.129:35091)
0619 17:01:14.156534 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:14.156568 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":65,"threads_started":1}
I20260619 17:01:14.195056 4233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:14.194731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.130:51765)
0619 17:01:14.194940 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:14.195007 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":65,"threads_started":1}
I20260619 17:01:14.199255 4234 raft_consensus.cc:482] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 17:01:14.199345 4234 raft_consensus.cc:504] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "86ed8d3ae791430d8acf249777ed79e1" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 42577 } } peers { permanent_uuid: "14686394078f408d8f2970b5a0218323" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 36657 } } peers { permanent_uuid: "82b7a7a6f8ce4fa798108c74c2b99ab4" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 38807 } }
I20260619 17:01:14.199527 4234 leader_election.cc:288] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 86ed8d3ae791430d8acf249777ed79e1 (127.20.231.130:42577), 14686394078f408d8f2970b5a0218323 (127.20.231.131:36657)
W20260619 17:01:14.199852 3812 leader_election.cc:341] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 86ed8d3ae791430d8acf249777ed79e1 (127.20.231.130:42577): Not found: Tablet not found: de40b7a929314c468cc3617774433870
W20260619 17:01:14.199982 3814 leader_election.cc:341] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 14686394078f408d8f2970b5a0218323 (127.20.231.131:36657): Not found: Tablet not found: de40b7a929314c468cc3617774433870
I20260619 17:01:14.200078 3814 leader_election.cc:302] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 82b7a7a6f8ce4fa798108c74c2b99ab4; no voters: 14686394078f408d8f2970b5a0218323, 86ed8d3ae791430d8acf249777ed79e1
I20260619 17:01:14.200165 4234 raft_consensus.cc:2726] T de40b7a929314c468cc3617774433870 P 82b7a7a6f8ce4fa798108c74c2b99ab4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 17:01:14.336932 4235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:14.336708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.131:35945)
0619 17:01:14.336848 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:14.336885 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":76,"threads_started":1}
I20260619 17:01:15.157501 4236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:15.157254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.129:54799)
0619 17:01:15.157412 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:15.157455 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":65,"threads_started":1}
I20260619 17:01:15.195771 4237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:15.195558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.130:60867)
0619 17:01:15.195690 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:15.195727 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":78,"threads_started":1}
I20260619 17:01:15.337617 4238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:15.337391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.131:49639)
0619 17:01:15.337528 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:15.337570 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":58,"threads_started":1}
I20260619 17:01:16.158291 4239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:16.158051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.129:45633)
0619 17:01:16.158196 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:16.158243 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":69,"threads_started":1}
I20260619 17:01:16.196488 4240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:16.196278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.130:44987)
0619 17:01:16.196405 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:16.196444 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":64,"threads_started":1}
I20260619 17:01:16.338456 4241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:16.338187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.131:58821)
0619 17:01:16.338333 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:16.338406 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":67,"threads_started":1}
I20260619 17:01:17.159302 4242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:17.158997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.129:48959)
0619 17:01:17.159133 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:17.159229 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":64,"threads_started":1}
I20260619 17:01:17.197268 4243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:17.197042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.130:50745)
0619 17:01:17.197182 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:17.197219 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":65,"threads_started":1}
I20260619 17:01:17.339432 4244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:17.339177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.131:35923)
0619 17:01:17.339329 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:17.339383 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":81,"threads_started":1}
I20260619 17:01:18.160296 4245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:18.160046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.129:34587)
0619 17:01:18.160193 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:18.160241 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":65,"threads_started":1}
I20260619 17:01:18.198307 4246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:18.197945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.130:54817)
0619 17:01:18.198191 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:18.198249 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":66,"threads_started":1}
I20260619 17:01:18.337131 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 3766
I20260619 17:01:18.340313 4247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:18.340045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33699 (local address 127.20.231.131:32863)
0619 17:01:18.340200 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:18.340250 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:33699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":63,"threads_started":1}
I20260619 17:01:18.342133 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 3930
I20260619 17:01:18.346436 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 4062
2026-06-19T17:01:18Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18602 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 17:01:18 dist-test-slave-vnbj krb5kdc[4250](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 17:01:18 dist-test-slave-vnbj krb5kdc[4250](info): set up 2 sockets
Jun 19 17:01:18 dist-test-slave-vnbj krb5kdc[4250](info): commencing operation
krb5kdc: starting...
W20260619 17:01:19.419080 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 17:01:19 dist-test-slave-vnbj krb5kdc[4250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888479, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T17:01:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T17:01:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.231.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
I20260619 17:01:19.550675 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:41493
--webserver_interface=127.20.231.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:43231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:41493
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:01:19.639879 4263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:01:19.640116 4263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:01:19.640141 4263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:01:19.641443 4263 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 17:01:19.641494 4263 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 17:01:19.641507 4263 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 17:01:19.641518 4263 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:01:19.641532 4263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:01:19.641546 4263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:01:19.641558 4263 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:01:19.641572 4263 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:01:19.643321 4263 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:43231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:41493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:41493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:01:19.643605 4263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:01:19.643853 4263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:01:19.649854 4271 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:01:19.649818 4269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:01:19.649883 4268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:01:19.649895 4263 server_base.cc:1034] running on GCE node
I20260619 17:01:19.650318 4263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:01:19.650668 4263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:01:19.651898 4263 hybrid_clock.cc:648] HybridClock initialized: now 1781888479651870 us; error 52 us; skew 500 ppm
Jun 19 17:01:19 dist-test-slave-vnbj krb5kdc[4250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888479, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:01:19.654702 4263 init.cc:377] Logged in from keytab as kudu/127.20.231.190@KRBTEST.COM (short username kudu)
I20260619 17:01:19.655328 4263 webserver.cc:460] Webserver started at http://127.20.231.190:33955/ using document root <none> and password file <none>
I20260619 17:01:19.655570 4263 fs_manager.cc:359] Metadata directory not provided
I20260619 17:01:19.655637 4263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:01:19.655812 4263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:01:19.656934 4263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "e32dbf7acea04f4a83288bb09da26c27"
format_stamp: "Formatted at 2026-06-19 17:01:19 on dist-test-slave-vnbj"
server_key: "918405ff9790103545cff23661c5ccd2"
server_key_iv: "aa71d0a836a4aa888a18580ee37a45eb"
server_key_version: "encryptionkey@0"
I20260619 17:01:19.657310 4263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "e32dbf7acea04f4a83288bb09da26c27"
format_stamp: "Formatted at 2026-06-19 17:01:19 on dist-test-slave-vnbj"
server_key: "918405ff9790103545cff23661c5ccd2"
server_key_iv: "aa71d0a836a4aa888a18580ee37a45eb"
server_key_version: "encryptionkey@0"
I20260619 17:01:19.658779 4263 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 17:01:19.659850 4278 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:19.660100 4263 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 17:01:19.660344 4263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "e32dbf7acea04f4a83288bb09da26c27"
format_stamp: "Formatted at 2026-06-19 17:01:19 on dist-test-slave-vnbj"
server_key: "918405ff9790103545cff23661c5ccd2"
server_key_iv: "aa71d0a836a4aa888a18580ee37a45eb"
server_key_version: "encryptionkey@0"
I20260619 17:01:19.660446 4263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:01:19.677983 4263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:01:19.680612 4263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:01:19.680868 4263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:01:19.685282 4263 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:41493
I20260619 17:01:19.685316 4330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:41493 every 8 connection(s)
I20260619 17:01:19.685688 4263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 17:01:19.686349 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 4263
I20260619 17:01:19.686482 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 17:01:19.686724 21406 external_mini_cluster.cc:1428] Setting key bbae2fd5bdba3a1f6fe5d81c4befe6f8
I20260619 17:01:19.686954 4331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:01:19.690829 4331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27: Bootstrap starting.
I20260619 17:01:19.691648 4331 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27: Neither blocks nor log segments found. Creating new log.
I20260619 17:01:19.692028 4331 log.cc:824] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27: Log is configured to *not* fsync() on all Append() calls
I20260619 17:01:19.692953 4331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27: No bootstrap required, opened a new log
Jun 19 17:01:19 dist-test-slave-vnbj krb5kdc[4250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888479, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:01:19.695349 4331 raft_consensus.cc:348] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e32dbf7acea04f4a83288bb09da26c27" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41493 } }
I20260619 17:01:19.695505 4331 raft_consensus.cc:374] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:01:19.695529 4331 raft_consensus.cc:729] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e32dbf7acea04f4a83288bb09da26c27, State: Initialized, Role: FOLLOWER
I20260619 17:01:19.695690 4331 consensus_queue.cc:260] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e32dbf7acea04f4a83288bb09da26c27" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41493 } }
I20260619 17:01:19.695755 4331 raft_consensus.cc:388] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 17:01:19.695797 4331 raft_consensus.cc:482] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 17:01:19.695879 4331 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:01:19.696466 4331 raft_consensus.cc:504] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e32dbf7acea04f4a83288bb09da26c27" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41493 } }
I20260619 17:01:19.696597 4331 leader_election.cc:302] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e32dbf7acea04f4a83288bb09da26c27; no voters:
I20260619 17:01:19.696802 4331 leader_election.cc:288] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 17:01:19.696834 4336 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:01:19.696939 4336 raft_consensus.cc:686] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [term 1 LEADER]: Becoming Leader. State: Replica: e32dbf7acea04f4a83288bb09da26c27, State: Running, Role: LEADER
I20260619 17:01:19.697079 4331 sys_catalog.cc:564] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:01:19.697077 4336 consensus_queue.cc:237] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e32dbf7acea04f4a83288bb09da26c27" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41493 } }
I20260619 17:01:19.698458 4334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:19.687437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43994 (local address 127.20.231.190:41493)
0619 17:01:19.687615 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:19.687632 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:19.687666 (+ 34us) server_negotiation.cc:407] Connection header received
0619 17:01:19.688196 (+ 530us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:19.688202 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:19.688252 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:19.688381 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:01:19.689356 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:19.690255 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:19.691460 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:19.691912 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:19.691992 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:19.694910 (+ 2918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:01:19.694916 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:01:19.694921 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:01:19.694933 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:01:19.696869 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:19.697261 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:19.697266 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:19.697266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:19.697374 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:19.697554 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:19.697555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:19.697556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:19.697827 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:01:19.697992 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:19.698128 (+ 136us) server_negotiation.cc:299] Negotiation successful
0619 17:01:19.698285 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":97,"threads_started":1}
I20260619 17:01:19.698985 4338 sys_catalog.cc:455] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e32dbf7acea04f4a83288bb09da26c27. Latest consensus state: current_term: 1 leader_uuid: "e32dbf7acea04f4a83288bb09da26c27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e32dbf7acea04f4a83288bb09da26c27" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41493 } } }
I20260619 17:01:19.699123 4337 sys_catalog.cc:455] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e32dbf7acea04f4a83288bb09da26c27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e32dbf7acea04f4a83288bb09da26c27" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41493 } } }
I20260619 17:01:19.699399 4338 sys_catalog.cc:458] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [sys.catalog]: This master's current role is: LEADER
I20260619 17:01:19.699409 4337 sys_catalog.cc:458] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [sys.catalog]: This master's current role is: LEADER
I20260619 17:01:19.699610 4353 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 17:01:19.700043 4353 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 17:01:19.701771 4353 catalog_manager.cc:1347] Generated new cluster ID: 9e590b6b1b1c4b0faa086a41bc51f959
I20260619 17:01:19.701823 4353 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 17:01:19.708983 4353 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 17:01:19.709618 4353 catalog_manager.cc:1504] Loading token signing keys...
I20260619 17:01:19.712605 4353 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27: Generated new TSK 0
I20260619 17:01:19.712877 4353 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.231.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab.
Entry for principal kudu/127.20.231.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab.
I20260619 17:01:19.750548 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.129:0
--local_ip_for_outbound_sockets=127.20.231.129
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:41493
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:43231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:01:19.854383 4358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:01:19.854609 4358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:01:19.854635 4358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:01:19.856011 4358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:01:19.856060 4358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:01:19.856089 4358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.129
I20260619 17:01:19.857671 4358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:43231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:41493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.129
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:01:19.857941 4358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:01:19.858202 4358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:01:19.864040 4364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:01:19.864030 4363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:01:19.864035 4366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:01:19.864075 4358 server_base.cc:1034] running on GCE node
I20260619 17:01:19.864542 4358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:01:19.864840 4358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:01:19.866031 4358 hybrid_clock.cc:648] HybridClock initialized: now 1781888479866000 us; error 49 us; skew 500 ppm
Jun 19 17:01:19 dist-test-slave-vnbj krb5kdc[4250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888479, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:01:19.869481 4358 init.cc:377] Logged in from keytab as kudu/127.20.231.129@KRBTEST.COM (short username kudu)
I20260619 17:01:19.869943 4358 webserver.cc:460] Webserver started at http://127.20.231.129:43213/ using document root <none> and password file <none>
I20260619 17:01:19.870249 4358 fs_manager.cc:359] Metadata directory not provided
I20260619 17:01:19.870328 4358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:01:19.870486 4358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:01:19.871708 4358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/data/instance:
uuid: "ea83fb233f30420b8639ba14498653c2"
format_stamp: "Formatted at 2026-06-19 17:01:19 on dist-test-slave-vnbj"
server_key: "98d201fecad496ed9a77e9331febb554"
server_key_iv: "c906756b4af81417da67a7b2598c1f28"
server_key_version: "encryptionkey@0"
I20260619 17:01:19.872083 4358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance:
uuid: "ea83fb233f30420b8639ba14498653c2"
format_stamp: "Formatted at 2026-06-19 17:01:19 on dist-test-slave-vnbj"
server_key: "98d201fecad496ed9a77e9331febb554"
server_key_iv: "c906756b4af81417da67a7b2598c1f28"
server_key_version: "encryptionkey@0"
I20260619 17:01:19.873541 4358 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 17:01:19.874518 4373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:19.874717 4358 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:01:19.874840 4358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/wal
uuid: "ea83fb233f30420b8639ba14498653c2"
format_stamp: "Formatted at 2026-06-19 17:01:19 on dist-test-slave-vnbj"
server_key: "98d201fecad496ed9a77e9331febb554"
server_key_iv: "c906756b4af81417da67a7b2598c1f28"
server_key_version: "encryptionkey@0"
I20260619 17:01:19.874959 4358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:01:19.886579 4358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:01:19.889256 4358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:01:19.889470 4358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:01:19.889748 4358 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:01:19.890108 4358 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:01:19.890168 4358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:19.890204 4358 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:01:19.890232 4358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:19.896761 4358 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.129:41923
I20260619 17:01:19.896800 4486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.129:41923 every 8 connection(s)
I20260619 17:01:19.897202 4358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
Jun 19 17:01:19 dist-test-slave-vnbj krb5kdc[4250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888479, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:01:19.907397 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 4358
I20260619 17:01:19.907532 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance
I20260619 17:01:19.907783 21406 external_mini_cluster.cc:1428] Setting key b2f82bd4e0febcc7b05dc31935c19f7e
I20260619 17:01:19.909670 4334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:19.898031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:58203 (local address 127.20.231.190:41493)
0619 17:01:19.898106 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:19.898108 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:19.898381 (+ 273us) server_negotiation.cc:407] Connection header received
0619 17:01:19.899368 (+ 987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:19.899370 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:19.899381 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:19.899442 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:01:19.900464 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:19.901081 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:19.902260 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:19.902507 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:19.902580 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:19.905495 (+ 2915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:01:19.905501 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:01:19.905502 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:01:19.905510 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:01:19.907582 (+ 2072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:19.908242 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:19.908243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:19.908244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:19.908324 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:19.908841 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:19.908843 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:19.908843 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:19.908997 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:01:19.909097 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:19.909398 (+ 301us) server_negotiation.cc:299] Negotiation successful
0619 17:01:19.909521 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 17:01:19.909716 4489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:19.898016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.129:58203)
0619 17:01:19.898196 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:19.898257 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0619 17:01:19.899151 (+ 894us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:01:19.899507 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:01:19.899510 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:01:19.900124 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:01:19.900334 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:19.900339 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:19.901220 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:19.901221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:19.902155 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:19.902158 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:19.902652 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:19.902653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:19.902955 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:19.903563 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:01:19.903581 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:01:19.905360 (+ 1779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:01:19.907697 (+ 2337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:01:19.907706 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:01:19.907709 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:01:19.908051 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:01:19.908449 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:01:19.908449 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:01:19.908450 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:01:19.908641 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:01:19.909084 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:01:19.909090 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:01:19.909249 (+ 159us) client_negotiation.cc:769] Sending connection context
0619 17:01:19.909351 (+ 102us) client_negotiation.cc:240] Negotiation successful
0619 17:01:19.909512 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":112,"threads_started":1}
I20260619 17:01:19.910429 4487 heartbeater.cc:347] Connected to a master server at 127.20.231.190:41493
I20260619 17:01:19.910565 4487 heartbeater.cc:464] Registering TS with master...
I20260619 17:01:19.910777 4487 heartbeater.cc:511] Master 127.20.231.190:41493 requested a full tablet report, sending...
I20260619 17:01:19.911474 4295 ts_manager.cc:194] Registered new tserver with Master: ea83fb233f30420b8639ba14498653c2 (127.20.231.129:41923)
I20260619 17:01:19.912199 4295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.129@KRBTEST.COM'} at 127.20.231.129:58203
WARNING: no policy specified for kudu/127.20.231.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab.
Entry for principal kudu/127.20.231.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab.
I20260619 17:01:19.939180 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.130:0
--local_ip_for_outbound_sockets=127.20.231.130
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:41493
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:43231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:01:20.023442 4492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:01:20.023721 4492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:01:20.023764 4492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:01:20.025049 4492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:01:20.025141 4492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:01:20.025190 4492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.130
I20260619 17:01:20.026803 4492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:43231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:41493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.130
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:01:20.027145 4492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:01:20.027412 4492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:01:20.032948 4498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:01:20.032986 4497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:01:20.032959 4500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:01:20.032984 4492 server_base.cc:1034] running on GCE node
I20260619 17:01:20.033385 4492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:01:20.033629 4492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:01:20.034785 4492 hybrid_clock.cc:648] HybridClock initialized: now 1781888480034759 us; error 44 us; skew 500 ppm
Jun 19 17:01:20 dist-test-slave-vnbj krb5kdc[4250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888480, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:01:20.037607 4492 init.cc:377] Logged in from keytab as kudu/127.20.231.130@KRBTEST.COM (short username kudu)
I20260619 17:01:20.038195 4492 webserver.cc:460] Webserver started at http://127.20.231.130:42209/ using document root <none> and password file <none>
I20260619 17:01:20.038470 4492 fs_manager.cc:359] Metadata directory not provided
I20260619 17:01:20.038544 4492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:01:20.038697 4492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:01:20.039943 4492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/data/instance:
uuid: "e45f8a53c1a54a84943d3641364ebc67"
format_stamp: "Formatted at 2026-06-19 17:01:20 on dist-test-slave-vnbj"
server_key: "34d862066aba9d978dc9fe26115eabf3"
server_key_iv: "51eb8d12855df27c15131e960fa1e49a"
server_key_version: "encryptionkey@0"
I20260619 17:01:20.040321 4492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance:
uuid: "e45f8a53c1a54a84943d3641364ebc67"
format_stamp: "Formatted at 2026-06-19 17:01:20 on dist-test-slave-vnbj"
server_key: "34d862066aba9d978dc9fe26115eabf3"
server_key_iv: "51eb8d12855df27c15131e960fa1e49a"
server_key_version: "encryptionkey@0"
I20260619 17:01:20.041742 4492 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 17:01:20.042661 4507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:20.042958 4492 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:01:20.043062 4492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/wal
uuid: "e45f8a53c1a54a84943d3641364ebc67"
format_stamp: "Formatted at 2026-06-19 17:01:20 on dist-test-slave-vnbj"
server_key: "34d862066aba9d978dc9fe26115eabf3"
server_key_iv: "51eb8d12855df27c15131e960fa1e49a"
server_key_version: "encryptionkey@0"
I20260619 17:01:20.043149 4492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:01:20.064782 4492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:01:20.067404 4492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:01:20.067603 4492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:01:20.067853 4492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:01:20.068245 4492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:01:20.068302 4492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:20.068346 4492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:01:20.068388 4492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:20.074548 4492 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.130:46227
I20260619 17:01:20.074592 4620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.130:46227 every 8 connection(s)
I20260619 17:01:20.075134 4492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
Jun 19 17:01:20 dist-test-slave-vnbj krb5kdc[4250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888480, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:01:20.084973 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 4492
I20260619 17:01:20.085112 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance
I20260619 17:01:20.085333 21406 external_mini_cluster.cc:1428] Setting key 1ef2482c4090b7bda7e3d40c3b7481d9
I20260619 17:01:20.086092 4334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:20.076044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:44637 (local address 127.20.231.190:41493)
0619 17:01:20.076115 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:20.076117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:20.076301 (+ 184us) server_negotiation.cc:407] Connection header received
0619 17:01:20.076825 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:20.076826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:20.076836 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:20.076905 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:01:20.077925 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:20.078624 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:20.079769 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:20.079996 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:20.080060 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:20.082599 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:01:20.082601 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:01:20.082601 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:01:20.082609 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:01:20.084155 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:20.084575 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:20.084576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:20.084577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:20.084631 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:20.085075 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:20.085077 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:20.085077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:20.085256 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:01:20.085390 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:20.085876 (+ 486us) server_negotiation.cc:299] Negotiation successful
0619 17:01:20.085986 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 17:01:20.086122 4623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:20.075997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.130:44637)
0619 17:01:20.076164 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:20.076209 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 17:01:20.076733 (+ 524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:01:20.076964 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:01:20.076968 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:01:20.077575 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:01:20.077796 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:20.077801 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:20.078751 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:20.078753 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:20.079674 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:20.079677 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:20.080084 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:20.080084 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:20.080194 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:20.080907 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:01:20.080916 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:01:20.082481 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:01:20.084249 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:01:20.084256 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:01:20.084258 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:01:20.084500 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:01:20.084812 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:01:20.084813 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:01:20.084813 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:01:20.084966 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:01:20.085430 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:01:20.085439 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:01:20.085606 (+ 167us) client_negotiation.cc:769] Sending connection context
0619 17:01:20.085761 (+ 155us) client_negotiation.cc:240] Negotiation successful
0619 17:01:20.085926 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":95,"threads_started":1}
I20260619 17:01:20.086784 4621 heartbeater.cc:347] Connected to a master server at 127.20.231.190:41493
I20260619 17:01:20.086951 4621 heartbeater.cc:464] Registering TS with master...
I20260619 17:01:20.087189 4621 heartbeater.cc:511] Master 127.20.231.190:41493 requested a full tablet report, sending...
I20260619 17:01:20.087661 4295 ts_manager.cc:194] Registered new tserver with Master: e45f8a53c1a54a84943d3641364ebc67 (127.20.231.130:46227)
I20260619 17:01:20.088208 4295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.130@KRBTEST.COM'} at 127.20.231.130:44637
WARNING: no policy specified for kudu/127.20.231.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab.
Entry for principal kudu/127.20.231.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab.
I20260619 17:01:20.114014 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.131:0
--local_ip_for_outbound_sockets=127.20.231.131
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:41493
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:43231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:01:20.203876 4626 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:01:20.204128 4626 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:01:20.204157 4626 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:01:20.205469 4626 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:01:20.205528 4626 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:01:20.205560 4626 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.131
I20260619 17:01:20.207230 4626 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:43231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:41493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4626
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.131
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:01:20.207502 4626 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:01:20.207774 4626 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:01:20.213318 4632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:01:20.213322 4631 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:01:20.213325 4634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:01:20.213454 4626 server_base.cc:1034] running on GCE node
I20260619 17:01:20.215230 4626 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:01:20.215525 4626 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:01:20.216789 4626 hybrid_clock.cc:648] HybridClock initialized: now 1781888480216744 us; error 57 us; skew 500 ppm
Jun 19 17:01:20 dist-test-slave-vnbj krb5kdc[4250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888480, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:01:20.220119 4626 init.cc:377] Logged in from keytab as kudu/127.20.231.131@KRBTEST.COM (short username kudu)
I20260619 17:01:20.220672 4626 webserver.cc:460] Webserver started at http://127.20.231.131:37707/ using document root <none> and password file <none>
I20260619 17:01:20.220906 4626 fs_manager.cc:359] Metadata directory not provided
I20260619 17:01:20.220988 4626 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:01:20.221104 4626 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:01:20.222252 4626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/data/instance:
uuid: "168e7085a6724eda937626d694c09a10"
format_stamp: "Formatted at 2026-06-19 17:01:20 on dist-test-slave-vnbj"
server_key: "84c4a271b887166f0a1169318be2ec48"
server_key_iv: "da296dbec863eec43149ecf3353e0ff0"
server_key_version: "encryptionkey@0"
I20260619 17:01:20.222596 4626 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance:
uuid: "168e7085a6724eda937626d694c09a10"
format_stamp: "Formatted at 2026-06-19 17:01:20 on dist-test-slave-vnbj"
server_key: "84c4a271b887166f0a1169318be2ec48"
server_key_iv: "da296dbec863eec43149ecf3353e0ff0"
server_key_version: "encryptionkey@0"
I20260619 17:01:20.224197 4626 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 17:01:20.225179 4641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:20.225423 4626 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:01:20.225533 4626 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/wal
uuid: "168e7085a6724eda937626d694c09a10"
format_stamp: "Formatted at 2026-06-19 17:01:20 on dist-test-slave-vnbj"
server_key: "84c4a271b887166f0a1169318be2ec48"
server_key_iv: "da296dbec863eec43149ecf3353e0ff0"
server_key_version: "encryptionkey@0"
I20260619 17:01:20.225600 4626 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:01:20.234007 4626 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:01:20.236490 4626 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:01:20.236640 4626 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:01:20.236847 4626 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:01:20.237115 4626 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:01:20.237147 4626 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:20.237171 4626 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:01:20.237185 4626 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:20.243541 4626 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.131:40067
I20260619 17:01:20.243657 4754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.131:40067 every 8 connection(s)
I20260619 17:01:20.243969 4626 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
I20260619 17:01:20.250011 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 4626
I20260619 17:01:20.250088 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance
I20260619 17:01:20.250288 21406 external_mini_cluster.cc:1428] Setting key aeee885b92ad3c45203b431ba1c8c662
Jun 19 17:01:20 dist-test-slave-vnbj krb5kdc[4250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888480, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:01:20.254640 4334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:20.244872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:35705 (local address 127.20.231.190:41493)
0619 17:01:20.244957 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:20.244959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:20.245141 (+ 182us) server_negotiation.cc:407] Connection header received
0619 17:01:20.245756 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:20.245758 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:20.245773 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:20.245832 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:01:20.247002 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:20.247514 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:20.248544 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:20.248761 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:20.248851 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:20.251521 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:01:20.251522 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:01:20.251522 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:01:20.251529 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:01:20.253089 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:20.253573 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:20.253574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:20.253575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:20.253635 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:20.254003 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:20.254004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:20.254004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:20.254122 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:01:20.254183 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:20.254489 (+ 306us) server_negotiation.cc:299] Negotiation successful
0619 17:01:20.254571 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 17:01:20.254947 4757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:20.244711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.131:35705)
0619 17:01:20.244995 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:20.245046 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 17:01:20.245567 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:01:20.245907 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:01:20.245911 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:01:20.246620 (+ 709us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:01:20.246829 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:20.246834 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:20.247613 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:20.247614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:20.248444 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:20.248446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:20.248861 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:20.248862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:20.249001 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:20.249610 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:01:20.249618 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:01:20.251440 (+ 1822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:01:20.253179 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:01:20.253185 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:01:20.253187 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:01:20.253463 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:01:20.253810 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:01:20.253811 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:01:20.253812 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:01:20.253933 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:01:20.254184 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:01:20.254188 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:01:20.254335 (+ 147us) client_negotiation.cc:769] Sending connection context
0619 17:01:20.254429 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 17:01:20.254569 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":100,"threads_started":1}
I20260619 17:01:20.255575 4755 heartbeater.cc:347] Connected to a master server at 127.20.231.190:41493
I20260619 17:01:20.255705 4755 heartbeater.cc:464] Registering TS with master...
I20260619 17:01:20.255926 4755 heartbeater.cc:511] Master 127.20.231.190:41493 requested a full tablet report, sending...
I20260619 17:01:20.256366 4295 ts_manager.cc:194] Registered new tserver with Master: 168e7085a6724eda937626d694c09a10 (127.20.231.131:40067)
I20260619 17:01:20.256738 4295 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.131@KRBTEST.COM'} at 127.20.231.131:35705
I20260619 17:01:20.262496 21406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 17:01:20.270563 4334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:20.264052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43998 (local address 127.20.231.190:41493)
0619 17:01:20.264112 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:20.264113 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:20.264164 (+ 51us) server_negotiation.cc:407] Connection header received
0619 17:01:20.264221 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:20.264222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:20.264233 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:20.264289 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:01:20.264994 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:20.265505 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:20.266300 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:20.266485 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:20.266578 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:20.267867 (+ 1289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:01:20.267869 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:01:20.267869 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:01:20.267877 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:01:20.269304 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:20.269663 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:20.269664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:20.269665 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:20.269801 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:20.270022 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:20.270023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:20.270023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:20.270181 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:01:20.270281 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:20.270367 (+ 86us) server_negotiation.cc:299] Negotiation successful
0619 17:01:20.270480 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 17:01:20.272364 4295 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43998:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 17:01:20.273942 4295 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 17:01:20.281281 4768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:20.276705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:41923 (local address 127.0.0.1:36014)
0619 17:01:20.277240 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:20.277250 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 17:01:20.277303 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:01:20.277607 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:01:20.277611 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:01:20.277614 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:01:20.277693 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:20.277696 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:20.278792 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:20.278792 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:20.280148 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:20.280151 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:20.280917 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:20.280918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:20.280998 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:20.281013 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 17:01:20.281111 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 17:01:20.281172 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"mutex_wait_us":1,"thread_start_us":92,"threads_started":1}
I20260619 17:01:20.281363 4769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:20.276872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55314 (local address 127.20.231.131:40067)
0619 17:01:20.277429 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:20.277435 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:20.277445 (+ 10us) server_negotiation.cc:407] Connection header received
0619 17:01:20.277481 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:20.277483 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:20.277498 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:20.277811 (+ 313us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:01:20.277889 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:20.279035 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:20.280235 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:20.280999 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:20.281083 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:20.281138 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:20.281232 (+ 94us) server_negotiation.cc:299] Negotiation successful
0619 17:01:20.281270 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":516,"thread_start_us":116,"threads_started":1}
I20260619 17:01:20.281641 4771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:20.277048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32996 (local address 127.20.231.130:46227)
0619 17:01:20.277241 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:20.277244 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:20.277253 (+ 9us) server_negotiation.cc:407] Connection header received
0619 17:01:20.277382 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:20.277385 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:20.277405 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:20.277496 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:01:20.277803 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:20.279019 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:20.280313 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:20.281016 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:20.281083 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:20.281138 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:20.281520 (+ 382us) server_negotiation.cc:299] Negotiation successful
0619 17:01:20.281565 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":103,"threads_started":1}
I20260619 17:01:20.281865 4767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:20.276702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:46227 (local address 127.0.0.1:32996)
0619 17:01:20.277148 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:20.277159 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 17:01:20.277214 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:01:20.277638 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:01:20.277639 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:01:20.277642 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:01:20.277719 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:20.277721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:20.279133 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:20.279134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:20.280242 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:20.280244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:20.281315 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:20.281315 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:20.281373 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:20.281376 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 17:01:20.281746 (+ 370us) client_negotiation.cc:240] Negotiation successful
0619 17:01:20.281793 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"mutex_wait_us":27,"spinlock_wait_cycles":768,"thread_start_us":185,"threads_started":1}
I20260619 17:01:20.282088 4770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:20.276957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36014 (local address 127.20.231.129:41923)
0619 17:01:20.277127 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:20.277132 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:20.277336 (+ 204us) server_negotiation.cc:407] Connection header received
0619 17:01:20.277391 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:20.277394 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:20.277411 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:20.277505 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:01:20.277810 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:20.278617 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:20.280262 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:20.280812 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:20.280917 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:20.280984 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:20.281935 (+ 951us) server_negotiation.cc:299] Negotiation successful
0619 17:01:20.281999 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":91,"threads_started":1}
I20260619 17:01:20.281406 4766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:20.276697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:40067 (local address 127.0.0.1:55314)
0619 17:01:20.276812 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:20.276862 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 17:01:20.276932 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:01:20.277643 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:01:20.277644 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:01:20.277645 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:01:20.277719 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:20.277722 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:20.279160 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:20.279161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:20.280141 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:20.280144 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:20.281087 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:20.281088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:20.281147 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:20.281150 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 17:01:20.281215 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 17:01:20.281246 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"mutex_wait_us":2,"thread_start_us":69,"threads_started":1}
I20260619 17:01:20.282639 4555 tablet_service.cc:1467] Processing CreateTablet for tablet e5952087f8514dd29b186d18f9f63a42 (DEFAULT_TABLE table=test-table [id=f06122125f574e1c8d4a2465dc5688ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 17:01:20.282923 4689 tablet_service.cc:1467] Processing CreateTablet for tablet e5952087f8514dd29b186d18f9f63a42 (DEFAULT_TABLE table=test-table [id=f06122125f574e1c8d4a2465dc5688ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 17:01:20.283056 4555 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5952087f8514dd29b186d18f9f63a42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:01:20.283293 4689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5952087f8514dd29b186d18f9f63a42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:01:20.283270 4421 tablet_service.cc:1467] Processing CreateTablet for tablet e5952087f8514dd29b186d18f9f63a42 (DEFAULT_TABLE table=test-table [id=f06122125f574e1c8d4a2465dc5688ed]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 17:01:20.283576 4421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5952087f8514dd29b186d18f9f63a42. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:01:20.286085 4772 tablet_bootstrap.cc:492] T e5952087f8514dd29b186d18f9f63a42 P 168e7085a6724eda937626d694c09a10: Bootstrap starting.
I20260619 17:01:20.286665 4773 tablet_bootstrap.cc:492] T e5952087f8514dd29b186d18f9f63a42 P e45f8a53c1a54a84943d3641364ebc67: Bootstrap starting.
I20260619 17:01:20.286865 4772 tablet_bootstrap.cc:654] T e5952087f8514dd29b186d18f9f63a42 P 168e7085a6724eda937626d694c09a10: Neither blocks nor log segments found. Creating new log.
I20260619 17:01:20.286940 4774 tablet_bootstrap.cc:492] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2: Bootstrap starting.
I20260619 17:01:20.287566 4774 tablet_bootstrap.cc:654] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2: Neither blocks nor log segments found. Creating new log.
I20260619 17:01:20.287585 4772 log.cc:824] T e5952087f8514dd29b186d18f9f63a42 P 168e7085a6724eda937626d694c09a10: Log is configured to *not* fsync() on all Append() calls
I20260619 17:01:20.287635 4773 tablet_bootstrap.cc:654] T e5952087f8514dd29b186d18f9f63a42 P e45f8a53c1a54a84943d3641364ebc67: Neither blocks nor log segments found. Creating new log.
I20260619 17:01:20.288213 4774 log.cc:824] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2: Log is configured to *not* fsync() on all Append() calls
I20260619 17:01:20.288213 4773 log.cc:824] T e5952087f8514dd29b186d18f9f63a42 P e45f8a53c1a54a84943d3641364ebc67: Log is configured to *not* fsync() on all Append() calls
I20260619 17:01:20.288465 4772 tablet_bootstrap.cc:492] T e5952087f8514dd29b186d18f9f63a42 P 168e7085a6724eda937626d694c09a10: No bootstrap required, opened a new log
I20260619 17:01:20.288568 4772 ts_tablet_manager.cc:1397] T e5952087f8514dd29b186d18f9f63a42 P 168e7085a6724eda937626d694c09a10: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 17:01:20.289004 4774 tablet_bootstrap.cc:492] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2: No bootstrap required, opened a new log
I20260619 17:01:20.289088 4774 ts_tablet_manager.cc:1397] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 17:01:20.289227 4773 tablet_bootstrap.cc:492] T e5952087f8514dd29b186d18f9f63a42 P e45f8a53c1a54a84943d3641364ebc67: No bootstrap required, opened a new log
I20260619 17:01:20.289299 4773 ts_tablet_manager.cc:1397] T e5952087f8514dd29b186d18f9f63a42 P e45f8a53c1a54a84943d3641364ebc67: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 17:01:20.290623 4772 raft_consensus.cc:348] T e5952087f8514dd29b186d18f9f63a42 P 168e7085a6724eda937626d694c09a10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e45f8a53c1a54a84943d3641364ebc67" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46227 } } peers { permanent_uuid: "168e7085a6724eda937626d694c09a10" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40067 } } peers { permanent_uuid: "ea83fb233f30420b8639ba14498653c2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 41923 } }
I20260619 17:01:20.290834 4772 raft_consensus.cc:374] T e5952087f8514dd29b186d18f9f63a42 P 168e7085a6724eda937626d694c09a10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:01:20.290956 4772 raft_consensus.cc:729] T e5952087f8514dd29b186d18f9f63a42 P 168e7085a6724eda937626d694c09a10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 168e7085a6724eda937626d694c09a10, State: Initialized, Role: FOLLOWER
I20260619 17:01:20.290781 4774 raft_consensus.cc:348] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e45f8a53c1a54a84943d3641364ebc67" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46227 } } peers { permanent_uuid: "168e7085a6724eda937626d694c09a10" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40067 } } peers { permanent_uuid: "ea83fb233f30420b8639ba14498653c2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 41923 } }
I20260619 17:01:20.291010 4774 raft_consensus.cc:374] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:01:20.291077 4774 raft_consensus.cc:729] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea83fb233f30420b8639ba14498653c2, State: Initialized, Role: FOLLOWER
I20260619 17:01:20.291119 4772 consensus_queue.cc:260] T e5952087f8514dd29b186d18f9f63a42 P 168e7085a6724eda937626d694c09a10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e45f8a53c1a54a84943d3641364ebc67" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46227 } } peers { permanent_uuid: "168e7085a6724eda937626d694c09a10" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40067 } } peers { permanent_uuid: "ea83fb233f30420b8639ba14498653c2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 41923 } }
I20260619 17:01:20.291201 4774 consensus_queue.cc:260] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e45f8a53c1a54a84943d3641364ebc67" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46227 } } peers { permanent_uuid: "168e7085a6724eda937626d694c09a10" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40067 } } peers { permanent_uuid: "ea83fb233f30420b8639ba14498653c2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 41923 } }
I20260619 17:01:20.291500 4772 ts_tablet_manager.cc:1428] T e5952087f8514dd29b186d18f9f63a42 P 168e7085a6724eda937626d694c09a10: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260619 17:01:20.291563 4774 ts_tablet_manager.cc:1428] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 17:01:20.291455 4773 raft_consensus.cc:348] T e5952087f8514dd29b186d18f9f63a42 P e45f8a53c1a54a84943d3641364ebc67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e45f8a53c1a54a84943d3641364ebc67" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46227 } } peers { permanent_uuid: "168e7085a6724eda937626d694c09a10" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40067 } } peers { permanent_uuid: "ea83fb233f30420b8639ba14498653c2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 41923 } }
I20260619 17:01:20.291509 4755 heartbeater.cc:503] Master 127.20.231.190:41493 was elected leader, sending a full tablet report...
I20260619 17:01:20.291615 4773 raft_consensus.cc:374] T e5952087f8514dd29b186d18f9f63a42 P e45f8a53c1a54a84943d3641364ebc67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:01:20.291682 4487 heartbeater.cc:503] Master 127.20.231.190:41493 was elected leader, sending a full tablet report...
I20260619 17:01:20.291676 4773 raft_consensus.cc:729] T e5952087f8514dd29b186d18f9f63a42 P e45f8a53c1a54a84943d3641364ebc67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e45f8a53c1a54a84943d3641364ebc67, State: Initialized, Role: FOLLOWER
I20260619 17:01:20.291910 4773 consensus_queue.cc:260] T e5952087f8514dd29b186d18f9f63a42 P e45f8a53c1a54a84943d3641364ebc67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e45f8a53c1a54a84943d3641364ebc67" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46227 } } peers { permanent_uuid: "168e7085a6724eda937626d694c09a10" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40067 } } peers { permanent_uuid: "ea83fb233f30420b8639ba14498653c2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 41923 } }
I20260619 17:01:20.292207 4621 heartbeater.cc:503] Master 127.20.231.190:41493 was elected leader, sending a full tablet report...
I20260619 17:01:20.292205 4773 ts_tablet_manager.cc:1428] T e5952087f8514dd29b186d18f9f63a42 P e45f8a53c1a54a84943d3641364ebc67: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260619 17:01:20.325868 4622 tablet.cc:2369] T e5952087f8514dd29b186d18f9f63a42 P e45f8a53c1a54a84943d3641364ebc67: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 17:01:20.355530 4779 raft_consensus.cc:482] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 17:01:20.355773 4779 raft_consensus.cc:504] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e45f8a53c1a54a84943d3641364ebc67" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46227 } } peers { permanent_uuid: "168e7085a6724eda937626d694c09a10" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40067 } } peers { permanent_uuid: "ea83fb233f30420b8639ba14498653c2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 41923 } }
I20260619 17:01:20.356264 4779 leader_election.cc:288] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e45f8a53c1a54a84943d3641364ebc67 (127.20.231.130:46227), 168e7085a6724eda937626d694c09a10 (127.20.231.131:40067)
I20260619 17:01:20.360519 4769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:20.356593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:33835 (local address 127.20.231.131:40067)
0619 17:01:20.356638 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:20.356640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:20.356802 (+ 162us) server_negotiation.cc:407] Connection header received
0619 17:01:20.356841 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:20.356842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:20.356851 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:20.356910 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:01:20.357038 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:20.357836 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:20.359442 (+ 1606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:20.360077 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:20.360172 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:20.360182 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:20.360402 (+ 220us) server_negotiation.cc:299] Negotiation successful
0619 17:01:20.360446 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 17:01:20.360519 4771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:20.356499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:52673 (local address 127.20.231.130:46227)
0619 17:01:20.356552 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:20.356555 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:20.356565 (+ 10us) server_negotiation.cc:407] Connection header received
0619 17:01:20.356624 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:20.356625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:20.356647 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:20.356725 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:01:20.356981 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:20.357837 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:20.359465 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:20.360076 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:20.360172 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:20.360182 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:20.360403 (+ 221us) server_negotiation.cc:299] Negotiation successful
0619 17:01:20.360446 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 17:01:20.360527 4489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:20.356435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:46227 (local address 127.20.231.129:52673)
0619 17:01:20.356513 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:20.356525 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 17:01:20.356567 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:01:20.356760 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:01:20.356761 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:01:20.356763 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:01:20.356845 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:20.356846 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:20.357991 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:20.357992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:20.359303 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:20.359306 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:20.360193 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:20.360194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:20.360290 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:20.360293 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 17:01:20.360371 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 17:01:20.360424 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":11}
I20260619 17:01:20.360534 4781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:20.356475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:40067 (local address 127.20.231.129:33835)
0619 17:01:20.356667 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:20.356684 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 17:01:20.356776 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:01:20.356928 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:01:20.356929 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:01:20.356932 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:01:20.356981 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:20.356982 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:20.358078 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:20.358079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:20.359303 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:20.359306 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:20.360193 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:20.360194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:20.360288 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:20.360293 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 17:01:20.360385 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 17:01:20.360422 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":8,"thread_start_us":128,"threads_started":1}
I20260619 17:01:20.361019 4575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5952087f8514dd29b186d18f9f63a42" candidate_uuid: "ea83fb233f30420b8639ba14498653c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e45f8a53c1a54a84943d3641364ebc67" is_pre_election: true
I20260619 17:01:20.361027 4709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5952087f8514dd29b186d18f9f63a42" candidate_uuid: "ea83fb233f30420b8639ba14498653c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "168e7085a6724eda937626d694c09a10" is_pre_election: true
I20260619 17:01:20.361191 4575 raft_consensus.cc:2454] T e5952087f8514dd29b186d18f9f63a42 P e45f8a53c1a54a84943d3641364ebc67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea83fb233f30420b8639ba14498653c2 in term 0.
I20260619 17:01:20.361191 4709 raft_consensus.cc:2454] T e5952087f8514dd29b186d18f9f63a42 P 168e7085a6724eda937626d694c09a10 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea83fb233f30420b8639ba14498653c2 in term 0.
I20260619 17:01:20.361446 4375 leader_election.cc:302] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e45f8a53c1a54a84943d3641364ebc67, ea83fb233f30420b8639ba14498653c2; no voters:
I20260619 17:01:20.361577 4779 raft_consensus.cc:2781] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 17:01:20.361632 4779 raft_consensus.cc:482] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 17:01:20.361667 4779 raft_consensus.cc:3037] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:01:20.362429 4779 raft_consensus.cc:504] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e45f8a53c1a54a84943d3641364ebc67" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46227 } } peers { permanent_uuid: "168e7085a6724eda937626d694c09a10" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40067 } } peers { permanent_uuid: "ea83fb233f30420b8639ba14498653c2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 41923 } }
I20260619 17:01:20.362802 4779 leader_election.cc:288] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2 [CANDIDATE]: Term 1 election: Requested vote from peers e45f8a53c1a54a84943d3641364ebc67 (127.20.231.130:46227), 168e7085a6724eda937626d694c09a10 (127.20.231.131:40067)
I20260619 17:01:20.362982 4575 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5952087f8514dd29b186d18f9f63a42" candidate_uuid: "ea83fb233f30420b8639ba14498653c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e45f8a53c1a54a84943d3641364ebc67"
I20260619 17:01:20.363083 4575 raft_consensus.cc:3037] T e5952087f8514dd29b186d18f9f63a42 P e45f8a53c1a54a84943d3641364ebc67 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:01:20.363080 4709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5952087f8514dd29b186d18f9f63a42" candidate_uuid: "ea83fb233f30420b8639ba14498653c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "168e7085a6724eda937626d694c09a10"
I20260619 17:01:20.363194 4709 raft_consensus.cc:3037] T e5952087f8514dd29b186d18f9f63a42 P 168e7085a6724eda937626d694c09a10 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:01:20.364158 4575 raft_consensus.cc:2454] T e5952087f8514dd29b186d18f9f63a42 P e45f8a53c1a54a84943d3641364ebc67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea83fb233f30420b8639ba14498653c2 in term 1.
I20260619 17:01:20.364390 4375 leader_election.cc:302] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e45f8a53c1a54a84943d3641364ebc67, ea83fb233f30420b8639ba14498653c2; no voters:
I20260619 17:01:20.364538 4779 raft_consensus.cc:2781] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:01:20.364611 4709 raft_consensus.cc:2454] T e5952087f8514dd29b186d18f9f63a42 P 168e7085a6724eda937626d694c09a10 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea83fb233f30420b8639ba14498653c2 in term 1.
I20260619 17:01:20.364753 4779 raft_consensus.cc:686] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2 [term 1 LEADER]: Becoming Leader. State: Replica: ea83fb233f30420b8639ba14498653c2, State: Running, Role: LEADER
I20260619 17:01:20.365054 4779 consensus_queue.cc:237] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e45f8a53c1a54a84943d3641364ebc67" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46227 } } peers { permanent_uuid: "168e7085a6724eda937626d694c09a10" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40067 } } peers { permanent_uuid: "ea83fb233f30420b8639ba14498653c2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 41923 } }
I20260619 17:01:20.366117 4295 catalog_manager.cc:5495] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to ea83fb233f30420b8639ba14498653c2 (127.20.231.129). New cstate: current_term: 1 leader_uuid: "ea83fb233f30420b8639ba14498653c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e45f8a53c1a54a84943d3641364ebc67" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "168e7085a6724eda937626d694c09a10" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40067 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea83fb233f30420b8639ba14498653c2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 41923 } health_report { overall_health: HEALTHY } } }
I20260619 17:01:20.380652 4770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:20.377468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36022 (local address 127.20.231.129:41923)
0619 17:01:20.377538 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:20.377540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:20.377547 (+ 7us) server_negotiation.cc:407] Connection header received
0619 17:01:20.377586 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:20.377587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:20.377607 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:20.377679 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 17:01:20.377930 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:20.378552 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:20.379701 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:20.379939 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:20.380027 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:20.380233 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 17:01:20.380367 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 17:01:20.380482 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:20.380502 (+ 20us) server_negotiation.cc:299] Negotiation successful
0619 17:01:20.380526 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 17:01:20.382648 4709 raft_consensus.cc:1261] T e5952087f8514dd29b186d18f9f63a42 P 168e7085a6724eda937626d694c09a10 [term 1 FOLLOWER]: Refusing update from remote peer ea83fb233f30420b8639ba14498653c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 17:01:20.382710 4575 raft_consensus.cc:1261] T e5952087f8514dd29b186d18f9f63a42 P e45f8a53c1a54a84943d3641364ebc67 [term 1 FOLLOWER]: Refusing update from remote peer ea83fb233f30420b8639ba14498653c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 17:01:20.383133 4779 consensus_queue.cc:1035] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e45f8a53c1a54a84943d3641364ebc67" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 17:01:20.383314 4782 consensus_queue.cc:1035] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "168e7085a6724eda937626d694c09a10" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40067 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 17:01:20.386009 4785 mvcc.cc:205] Tried to move back new op lower bound from 7298615215644520448 to 7298615215576035328. Current Snapshot: MvccSnapshot[applied={T|T < 7298615215644520448}]
I20260619 17:01:20.386312 4787 mvcc.cc:205] Tried to move back new op lower bound from 7298615215644520448 to 7298615215576035328. Current Snapshot: MvccSnapshot[applied={T|T < 7298615215644520448}]
I20260619 17:01:20.389405 4295 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 17:01:20.389533 4295 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 17:01:20.390447 4295 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27: Sending DeleteTablet for 3 replicas of tablet e5952087f8514dd29b186d18f9f63a42
I20260619 17:01:20.390870 4555 tablet_service.cc:1514] Processing DeleteTablet for tablet e5952087f8514dd29b186d18f9f63a42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:01:20 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:32996
I20260619 17:01:20.390928 4421 tablet_service.cc:1514] Processing DeleteTablet for tablet e5952087f8514dd29b186d18f9f63a42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:01:20 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:36014
I20260619 17:01:20.391142 4689 tablet_service.cc:1514] Processing DeleteTablet for tablet e5952087f8514dd29b186d18f9f63a42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:01:20 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:55314
I20260619 17:01:20.391415 4797 tablet_replica.cc:330] T e5952087f8514dd29b186d18f9f63a42 P 168e7085a6724eda937626d694c09a10: stopping tablet replica
I20260619 17:01:20.391474 4795 tablet_replica.cc:330] T e5952087f8514dd29b186d18f9f63a42 P e45f8a53c1a54a84943d3641364ebc67: stopping tablet replica
I20260619 17:01:20.391485 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 4358
I20260619 17:01:20.392277 4797 raft_consensus.cc:2229] T e5952087f8514dd29b186d18f9f63a42 P 168e7085a6724eda937626d694c09a10 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 17:01:20.392480 4797 raft_consensus.cc:2258] T e5952087f8514dd29b186d18f9f63a42 P 168e7085a6724eda937626d694c09a10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 17:01:20.392935 4797 ts_tablet_manager.cc:1905] T e5952087f8514dd29b186d18f9f63a42 P 168e7085a6724eda937626d694c09a10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 17:01:20.393486 4795 raft_consensus.cc:2229] T e5952087f8514dd29b186d18f9f63a42 P e45f8a53c1a54a84943d3641364ebc67 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 17:01:20.393630 4795 raft_consensus.cc:2258] T e5952087f8514dd29b186d18f9f63a42 P e45f8a53c1a54a84943d3641364ebc67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 17:01:20.394093 4795 ts_tablet_manager.cc:1905] T e5952087f8514dd29b186d18f9f63a42 P e45f8a53c1a54a84943d3641364ebc67: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 17:01:20.394809 4797 ts_tablet_manager.cc:1918] T e5952087f8514dd29b186d18f9f63a42 P 168e7085a6724eda937626d694c09a10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 17:01:20.395006 4797 log.cc:1196] T e5952087f8514dd29b186d18f9f63a42 P 168e7085a6724eda937626d694c09a10: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/wal/wals/e5952087f8514dd29b186d18f9f63a42
I20260619 17:01:20.395395 4797 ts_tablet_manager.cc:1939] T e5952087f8514dd29b186d18f9f63a42 P 168e7085a6724eda937626d694c09a10: Deleting consensus metadata
I20260619 17:01:20.395895 4281 catalog_manager.cc:4841] TS 168e7085a6724eda937626d694c09a10 (127.20.231.131:40067): tablet e5952087f8514dd29b186d18f9f63a42 (table test-table [id=f06122125f574e1c8d4a2465dc5688ed]) successfully deleted
I20260619 17:01:20.396152 4795 ts_tablet_manager.cc:1918] T e5952087f8514dd29b186d18f9f63a42 P e45f8a53c1a54a84943d3641364ebc67: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 17:01:20.396235 4795 log.cc:1196] T e5952087f8514dd29b186d18f9f63a42 P e45f8a53c1a54a84943d3641364ebc67: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/wal/wals/e5952087f8514dd29b186d18f9f63a42
I20260619 17:01:20.396515 4795 ts_tablet_manager.cc:1939] T e5952087f8514dd29b186d18f9f63a42 P e45f8a53c1a54a84943d3641364ebc67: Deleting consensus metadata
I20260619 17:01:20.397085 4280 catalog_manager.cc:4841] TS e45f8a53c1a54a84943d3641364ebc67 (127.20.231.130:46227): tablet e5952087f8514dd29b186d18f9f63a42 (table test-table [id=f06122125f574e1c8d4a2465dc5688ed]) successfully deleted
W20260619 17:01:20.398411 4279 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.231.129:41923 (error 108)
I20260619 17:01:20.398422 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 4492
I20260619 17:01:20.399812 4766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:20.399609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:41923 (local address 127.0.0.1:36034)
0619 17:01:20.399751 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:20.399784 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.129:41923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
W20260619 17:01:20.399991 4279 catalog_manager.cc:4584] TS ea83fb233f30420b8639ba14498653c2 (127.20.231.129:41923): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e5952087f8514dd29b186d18f9f63a42: Network error: Client connection negotiation failed: client connection to 127.20.231.129:41923: connect: Connection refused (error 111)
I20260619 17:01:20.403858 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 4626
I20260619 17:01:20.408877 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 4263
I20260619 17:01:40.414705 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:41493
--webserver_interface=127.20.231.190
--webserver_port=33955
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:43231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:41493
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:01:40.496723 4799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:01:40.496951 4799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:01:40.496974 4799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:01:40.498188 4799 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 17:01:40.498232 4799 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 17:01:40.498246 4799 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 17:01:40.498257 4799 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:01:40.498270 4799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:01:40.498283 4799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:01:40.498338 4799 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:01:40.498356 4799 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:01:40.499883 4799 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:43231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:41493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:41493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=33955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:01:40.500140 4799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:01:40.500386 4799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:01:40.509706 4805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:01:40.509750 4806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:01:40.509979 4799 server_base.cc:1034] running on GCE node
W20260619 17:01:40.510025 4808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:01:40.510318 4799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:01:40.510640 4799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:01:40.511816 4799 hybrid_clock.cc:648] HybridClock initialized: now 1781888500511788 us; error 62 us; skew 500 ppm
Jun 19 17:01:40 dist-test-slave-vnbj krb5kdc[4250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888500, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:01:40.514559 4799 init.cc:377] Logged in from keytab as kudu/127.20.231.190@KRBTEST.COM (short username kudu)
I20260619 17:01:40.515231 4799 webserver.cc:460] Webserver started at http://127.20.231.190:33955/ using document root <none> and password file <none>
I20260619 17:01:40.515507 4799 fs_manager.cc:359] Metadata directory not provided
I20260619 17:01:40.515619 4799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:01:40.517215 4799 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 17:01:40.517976 4815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:40.518307 4799 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 17:01:40.518425 4799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "e32dbf7acea04f4a83288bb09da26c27"
format_stamp: "Formatted at 2026-06-19 17:01:19 on dist-test-slave-vnbj"
server_key: "918405ff9790103545cff23661c5ccd2"
server_key_iv: "aa71d0a836a4aa888a18580ee37a45eb"
server_key_version: "encryptionkey@0"
I20260619 17:01:40.518975 4799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:01:40.535195 4799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:01:40.537683 4799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:01:40.537925 4799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:01:40.542140 4799 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:41493
I20260619 17:01:40.542182 4867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:41493 every 8 connection(s)
I20260619 17:01:40.542618 4799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 17:01:40.544469 4868 sys_catalog.cc:263] Verifying existing consensus state
I20260619 17:01:40.545288 4868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27: Bootstrap starting.
I20260619 17:01:40.550153 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 4799
I20260619 17:01:40.550493 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.129:41923
--local_ip_for_outbound_sockets=127.20.231.129
--tserver_master_addrs=127.20.231.190:41493
--webserver_port=43213
--webserver_interface=127.20.231.129
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:43231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 17:01:40.554934 4868 log.cc:824] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27: Log is configured to *not* fsync() on all Append() calls
I20260619 17:01:40.557057 4868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 17:01:40.557415 4868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27: Bootstrap complete.
I20260619 17:01:40.560081 4868 raft_consensus.cc:348] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e32dbf7acea04f4a83288bb09da26c27" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41493 } }
I20260619 17:01:40.560345 4868 raft_consensus.cc:729] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e32dbf7acea04f4a83288bb09da26c27, State: Initialized, Role: FOLLOWER
I20260619 17:01:40.560554 4868 consensus_queue.cc:260] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e32dbf7acea04f4a83288bb09da26c27" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41493 } }
I20260619 17:01:40.560623 4868 raft_consensus.cc:388] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 17:01:40.560694 4868 raft_consensus.cc:482] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 17:01:40.560755 4868 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [term 1 FOLLOWER]: Advancing to term 2
I20260619 17:01:40.561985 4868 raft_consensus.cc:504] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e32dbf7acea04f4a83288bb09da26c27" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41493 } }
I20260619 17:01:40.562117 4868 leader_election.cc:302] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e32dbf7acea04f4a83288bb09da26c27; no voters:
I20260619 17:01:40.562355 4868 leader_election.cc:288] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 17:01:40.562399 4872 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 17:01:40.562585 4872 raft_consensus.cc:686] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [term 2 LEADER]: Becoming Leader. State: Replica: e32dbf7acea04f4a83288bb09da26c27, State: Running, Role: LEADER
I20260619 17:01:40.562686 4868 sys_catalog.cc:564] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:01:40.562757 4872 consensus_queue.cc:237] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e32dbf7acea04f4a83288bb09da26c27" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41493 } }
I20260619 17:01:40.563102 4872 sys_catalog.cc:455] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e32dbf7acea04f4a83288bb09da26c27. Latest consensus state: current_term: 2 leader_uuid: "e32dbf7acea04f4a83288bb09da26c27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e32dbf7acea04f4a83288bb09da26c27" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41493 } } }
I20260619 17:01:40.563181 4872 sys_catalog.cc:458] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [sys.catalog]: This master's current role is: LEADER
I20260619 17:01:40.563128 4873 sys_catalog.cc:455] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e32dbf7acea04f4a83288bb09da26c27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e32dbf7acea04f4a83288bb09da26c27" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41493 } } }
I20260619 17:01:40.563446 4873 sys_catalog.cc:458] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [sys.catalog]: This master's current role is: LEADER
I20260619 17:01:40.563913 4876 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 17:01:40.564895 4876 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 17:01:40.565017 4876 catalog_manager.cc:1259] Loaded cluster ID: 9e590b6b1b1c4b0faa086a41bc51f959
I20260619 17:01:40.565055 4876 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 17:01:40.566188 4876 catalog_manager.cc:1504] Loading token signing keys...
I20260619 17:01:40.566411 4876 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27: Loaded TSK: 0
I20260619 17:01:40.574838 4876 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27: Generated new TSK 1
I20260619 17:01:40.574975 4876 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 17:01:40.645622 4870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:01:40.645867 4870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:01:40.645908 4870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:01:40.647115 4870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:01:40.647186 4870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:01:40.647230 4870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.129
I20260619 17:01:40.649356 4870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:43231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.129:41923
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.231.129
--webserver_port=43213
--tserver_master_addrs=127.20.231.190:41493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.129
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:01:40.649688 4870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:01:40.649982 4870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:01:40.655541 4896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:01:40.655606 4895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:01:40.655611 4898 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:01:40.656176 4870 server_base.cc:1034] running on GCE node
I20260619 17:01:40.656432 4870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:01:40.656694 4870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:01:40.657879 4870 hybrid_clock.cc:648] HybridClock initialized: now 1781888500657866 us; error 33 us; skew 500 ppm
Jun 19 17:01:40 dist-test-slave-vnbj krb5kdc[4250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888500, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:01:40.660622 4870 init.cc:377] Logged in from keytab as kudu/127.20.231.129@KRBTEST.COM (short username kudu)
I20260619 17:01:40.661163 4870 webserver.cc:460] Webserver started at http://127.20.231.129:43213/ using document root <none> and password file <none>
I20260619 17:01:40.661425 4870 fs_manager.cc:359] Metadata directory not provided
I20260619 17:01:40.661515 4870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:01:40.662973 4870 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 17:01:40.663784 4905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:40.664021 4870 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:01:40.664114 4870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/wal
uuid: "ea83fb233f30420b8639ba14498653c2"
format_stamp: "Formatted at 2026-06-19 17:01:19 on dist-test-slave-vnbj"
server_key: "98d201fecad496ed9a77e9331febb554"
server_key_iv: "c906756b4af81417da67a7b2598c1f28"
server_key_version: "encryptionkey@0"
I20260619 17:01:40.664614 4870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:01:40.678135 4870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:01:40.680658 4870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:01:40.680816 4870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:01:40.681066 4870 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:01:40.681546 4912 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 17:01:40.683295 4870 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 17:01:40.683357 4870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 17:01:40.683403 4870 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 17:01:40.684231 4870 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 17:01:40.684278 4870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 17:01:40.684360 4912 tablet_bootstrap.cc:492] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2: Bootstrap starting.
I20260619 17:01:40.691560 4870 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.129:41923
I20260619 17:01:40.691864 5019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.129:41923 every 8 connection(s)
I20260619 17:01:40.692011 4870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
I20260619 17:01:40.695556 4912 log.cc:824] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2: Log is configured to *not* fsync() on all Append() calls
I20260619 17:01:40.696414 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 4870
I20260619 17:01:40.696730 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.130:46227
--local_ip_for_outbound_sockets=127.20.231.130
--tserver_master_addrs=127.20.231.190:41493
--webserver_port=42209
--webserver_interface=127.20.231.130
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:43231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 17:01:40.698724 4912 tablet_bootstrap.cc:492] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 17:01:40.699069 4912 tablet_bootstrap.cc:492] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2: Bootstrap complete.
I20260619 17:01:40.700464 4912 ts_tablet_manager.cc:1397] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.007s
Jun 19 17:01:40 dist-test-slave-vnbj krb5kdc[4250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888500, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:01:40.702066 4912 raft_consensus.cc:348] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e45f8a53c1a54a84943d3641364ebc67" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46227 } } peers { permanent_uuid: "168e7085a6724eda937626d694c09a10" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40067 } } peers { permanent_uuid: "ea83fb233f30420b8639ba14498653c2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 41923 } }
I20260619 17:01:40.702410 4912 raft_consensus.cc:729] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea83fb233f30420b8639ba14498653c2, State: Initialized, Role: FOLLOWER
I20260619 17:01:40.702582 4912 consensus_queue.cc:260] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e45f8a53c1a54a84943d3641364ebc67" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46227 } } peers { permanent_uuid: "168e7085a6724eda937626d694c09a10" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40067 } } peers { permanent_uuid: "ea83fb233f30420b8639ba14498653c2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 41923 } }
I20260619 17:01:40.703298 4912 ts_tablet_manager.cc:1428] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2: Time spent starting tablet: real 0.003s user 0.003s sys 0.001s
I20260619 17:01:40.706619 5022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:40.692780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.129:60821)
0619 17:01:40.693294 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:40.693341 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 17:01:40.693959 (+ 618us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:01:40.694705 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:01:40.694709 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:01:40.695118 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:01:40.695388 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:40.695395 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:40.697041 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:40.697042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:40.698480 (+ 1438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:40.698484 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:40.699577 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:40.699578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:40.699743 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:40.700440 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:01:40.700451 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:01:40.702379 (+ 1928us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:01:40.704804 (+ 2425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:01:40.704811 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:01:40.704813 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:01:40.705057 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:01:40.705426 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:01:40.705427 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:01:40.705427 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:01:40.705537 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:01:40.705953 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:01:40.705960 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:01:40.706104 (+ 144us) client_negotiation.cc:769] Sending connection context
0619 17:01:40.706223 (+ 119us) client_negotiation.cc:240] Negotiation successful
0619 17:01:40.706410 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":90,"threads_started":1}
I20260619 17:01:40.706632 5023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:40.693110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:60821 (local address 127.20.231.190:41493)
0619 17:01:40.694171 (+ 1061us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:40.694180 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:40.694219 (+ 39us) server_negotiation.cc:407] Connection header received
0619 17:01:40.694358 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:40.694364 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:40.694430 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:40.694562 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:01:40.695523 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:40.696896 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:40.698583 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:40.699338 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:40.699463 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:40.702538 (+ 3075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:01:40.702544 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:01:40.702545 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:01:40.702557 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:01:40.704657 (+ 2100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:40.705178 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:40.705181 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:40.705182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:40.705335 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:40.705615 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:40.705617 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:40.705617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:40.705867 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:01:40.705963 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:40.706279 (+ 316us) server_negotiation.cc:299] Negotiation successful
0619 17:01:40.706421 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1010,"thread_start_us":78,"threads_started":1}
I20260619 17:01:40.707721 5020 heartbeater.cc:347] Connected to a master server at 127.20.231.190:41493
I20260619 17:01:40.707818 5020 heartbeater.cc:464] Registering TS with master...
I20260619 17:01:40.708014 5020 heartbeater.cc:511] Master 127.20.231.190:41493 requested a full tablet report, sending...
I20260619 17:01:40.708796 4832 ts_manager.cc:194] Registered new tserver with Master: ea83fb233f30420b8639ba14498653c2 (127.20.231.129:41923)
I20260619 17:01:40.710014 4832 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.129@KRBTEST.COM'} at 127.20.231.129:60821
I20260619 17:01:40.711045 5020 heartbeater.cc:503] Master 127.20.231.190:41493 was elected leader, sending a full tablet report...
I20260619 17:01:40.718386 5027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:40.709401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:41923 (local address 127.0.0.1:44444)
0619 17:01:40.715012 (+ 5611us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:40.715045 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 17:01:40.715105 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:01:40.715389 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:01:40.715391 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:01:40.715394 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:01:40.715462 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:40.715464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:40.716402 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:40.716403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:40.717288 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:40.717290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:40.718031 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:40.718032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:40.718119 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:40.718141 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 17:01:40.718240 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 17:01:40.718298 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5571,"thread_start_us":118,"threads_started":1}
I20260619 17:01:40.718379 5028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:40.709680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44444 (local address 127.20.231.129:41923)
0619 17:01:40.715217 (+ 5537us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:40.715221 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:40.715228 (+ 7us) server_negotiation.cc:407] Connection header received
0619 17:01:40.715256 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:40.715258 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:40.715270 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:40.715380 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:01:40.715548 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:40.716255 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:40.717409 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:40.717923 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:40.718030 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:40.718090 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:40.718258 (+ 168us) server_negotiation.cc:299] Negotiation successful
0619 17:01:40.718300 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5509,"thread_start_us":69,"threads_started":1}
I20260619 17:01:40.719144 4954 tablet_service.cc:1514] Processing DeleteTablet for tablet e5952087f8514dd29b186d18f9f63a42 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:01:20 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:44444
I20260619 17:01:40.719666 5029 tablet_replica.cc:330] stopping tablet replica
I20260619 17:01:40.719802 5029 raft_consensus.cc:2229] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 17:01:40.719882 5029 raft_consensus.cc:2258] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 17:01:40.720201 5029 ts_tablet_manager.cc:1905] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 17:01:40.722664 5029 ts_tablet_manager.cc:1918] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 17:01:40.722761 5029 log.cc:1196] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/wal/wals/e5952087f8514dd29b186d18f9f63a42
I20260619 17:01:40.723023 5029 ts_tablet_manager.cc:1939] T e5952087f8514dd29b186d18f9f63a42 P ea83fb233f30420b8639ba14498653c2: Deleting consensus metadata
I20260619 17:01:40.723555 4816 catalog_manager.cc:4841] TS ea83fb233f30420b8639ba14498653c2 (127.20.231.129:41923): tablet e5952087f8514dd29b186d18f9f63a42 (table test-table [id=f06122125f574e1c8d4a2465dc5688ed]) successfully deleted
W20260619 17:01:40.790205 5025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:01:40.790444 5025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:01:40.790473 5025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:01:40.791882 5025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:01:40.791942 5025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:01:40.791973 5025 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.130
I20260619 17:01:40.793668 5025 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:43231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.130:46227
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.231.130
--webserver_port=42209
--tserver_master_addrs=127.20.231.190:41493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.130
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:01:40.793949 5025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:01:40.794209 5025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:01:40.800014 5037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:01:40.800042 5034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:01:40.800045 5035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:01:40.800112 5025 server_base.cc:1034] running on GCE node
I20260619 17:01:40.800418 5025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:01:40.800699 5025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:01:40.801920 5025 hybrid_clock.cc:648] HybridClock initialized: now 1781888500801906 us; error 27 us; skew 500 ppm
Jun 19 17:01:40 dist-test-slave-vnbj krb5kdc[4250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888500, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:01:40.804682 5025 init.cc:377] Logged in from keytab as kudu/127.20.231.130@KRBTEST.COM (short username kudu)
I20260619 17:01:40.805084 5025 webserver.cc:460] Webserver started at http://127.20.231.130:42209/ using document root <none> and password file <none>
I20260619 17:01:40.805331 5025 fs_manager.cc:359] Metadata directory not provided
I20260619 17:01:40.805392 5025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:01:40.806780 5025 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:01:40.807514 5044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:40.807704 5025 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 17:01:40.807852 5025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/wal
uuid: "e45f8a53c1a54a84943d3641364ebc67"
format_stamp: "Formatted at 2026-06-19 17:01:20 on dist-test-slave-vnbj"
server_key: "34d862066aba9d978dc9fe26115eabf3"
server_key_iv: "51eb8d12855df27c15131e960fa1e49a"
server_key_version: "encryptionkey@0"
I20260619 17:01:40.808179 5025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:01:40.831856 5025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:01:40.834470 5025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:01:40.834604 5025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:01:40.834816 5025 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:01:40.835213 5025 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:01:40.835265 5025 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:40.835291 5025 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:01:40.835305 5025 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:40.842991 5025 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.130:46227
I20260619 17:01:40.843093 5157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.130:46227 every 8 connection(s)
I20260619 17:01:40.843482 5025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
I20260619 17:01:40.843878 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 5025
I20260619 17:01:40.844172 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.131:40067
--local_ip_for_outbound_sockets=127.20.231.131
--tserver_master_addrs=127.20.231.190:41493
--webserver_port=37707
--webserver_interface=127.20.231.131
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:43231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 17:01:40 dist-test-slave-vnbj krb5kdc[4250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888500, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:01:40.855074 5023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:40.844623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:44541 (local address 127.20.231.190:41493)
0619 17:01:40.844682 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:40.844684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:40.844702 (+ 18us) server_negotiation.cc:407] Connection header received
0619 17:01:40.845283 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:40.845285 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:40.845300 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:40.845367 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:01:40.846429 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:40.847401 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:40.848613 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:40.849053 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:40.849146 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:40.851558 (+ 2412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:01:40.851560 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:01:40.851561 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:01:40.851571 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:01:40.853511 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:40.853931 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:40.853933 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:40.853933 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:40.854005 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:40.854252 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:40.854253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:40.854253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:40.854442 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:01:40.854529 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:40.854790 (+ 261us) server_negotiation.cc:299] Negotiation successful
0619 17:01:40.854925 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 17:01:40.855108 5160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:40.844349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.130:44541)
0619 17:01:40.844547 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:40.844617 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0619 17:01:40.845154 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:01:40.845449 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:01:40.845453 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:01:40.846079 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:01:40.846311 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:40.846316 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:40.847510 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:40.847511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:40.848486 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:40.848489 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:40.849160 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:40.849161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:40.849273 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:40.849867 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:01:40.849877 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:01:40.851445 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:01:40.853610 (+ 2165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:01:40.853616 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:01:40.853618 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:01:40.853847 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:01:40.854082 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:01:40.854083 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:01:40.854083 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:01:40.854182 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:01:40.854523 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:01:40.854528 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:01:40.854648 (+ 120us) client_negotiation.cc:769] Sending connection context
0619 17:01:40.854730 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 17:01:40.854925 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":88,"threads_started":1}
I20260619 17:01:40.855783 5158 heartbeater.cc:347] Connected to a master server at 127.20.231.190:41493
I20260619 17:01:40.855921 5158 heartbeater.cc:464] Registering TS with master...
I20260619 17:01:40.856120 5158 heartbeater.cc:511] Master 127.20.231.190:41493 requested a full tablet report, sending...
I20260619 17:01:40.856639 4832 ts_manager.cc:194] Registered new tserver with Master: e45f8a53c1a54a84943d3641364ebc67 (127.20.231.130:46227)
I20260619 17:01:40.857048 4832 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.130@KRBTEST.COM'} at 127.20.231.130:44541
W20260619 17:01:40.932731 5161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:01:40.932987 5161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:01:40.933024 5161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:01:40.934226 5161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:01:40.934298 5161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:01:40.934338 5161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.131
I20260619 17:01:40.935851 5161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:43231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.131:40067
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.231.131
--webserver_port=37707
--tserver_master_addrs=127.20.231.190:41493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.131
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:01:40.936137 5161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:01:40.936388 5161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:01:40.941656 5166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:01:40.941672 5169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:01:40.941672 5167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:01:40.941880 5161 server_base.cc:1034] running on GCE node
I20260619 17:01:40.942211 5161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:01:40.942509 5161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:01:40.943759 5161 hybrid_clock.cc:648] HybridClock initialized: now 1781888500943682 us; error 61 us; skew 500 ppm
Jun 19 17:01:40 dist-test-slave-vnbj krb5kdc[4250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888500, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:01:40.946571 5161 init.cc:377] Logged in from keytab as kudu/127.20.231.131@KRBTEST.COM (short username kudu)
I20260619 17:01:40.947193 5161 webserver.cc:460] Webserver started at http://127.20.231.131:37707/ using document root <none> and password file <none>
I20260619 17:01:40.947475 5161 fs_manager.cc:359] Metadata directory not provided
I20260619 17:01:40.947563 5161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:01:40.950167 5161 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260619 17:01:40.950968 5176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:40.951150 5161 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:01:40.951270 5161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/wal
uuid: "168e7085a6724eda937626d694c09a10"
format_stamp: "Formatted at 2026-06-19 17:01:20 on dist-test-slave-vnbj"
server_key: "84c4a271b887166f0a1169318be2ec48"
server_key_iv: "da296dbec863eec43149ecf3353e0ff0"
server_key_version: "encryptionkey@0"
I20260619 17:01:40.951730 5161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:01:40.969038 5161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:01:40.971654 5161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:01:40.971833 5161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:01:40.972110 5161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:01:40.972468 5161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:01:40.972525 5161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:40.972561 5161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:01:40.972607 5161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:01:40.978592 5161 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.131:40067
I20260619 17:01:40.978631 5289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.131:40067 every 8 connection(s)
I20260619 17:01:40.979128 5161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
I20260619 17:01:40.980134 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 5161
Jun 19 17:01:40 dist-test-slave-vnbj krb5kdc[4250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888500, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:01:40.988528 5294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:40.980616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58014 (local address 127.20.231.190:41493)
0619 17:01:40.980759 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:40.980761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:40.980888 (+ 127us) server_negotiation.cc:407] Connection header received
0619 17:01:40.980966 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:40.980967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:40.980980 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:40.981040 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:01:40.981686 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:40.982301 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:40.983378 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:40.983687 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:40.983764 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:40.985299 (+ 1535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:01:40.985300 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:01:40.985301 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:01:40.985308 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:01:40.987045 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:40.987486 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:40.987487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:40.987487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:40.987552 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:40.987756 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:40.987757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:40.987757 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:40.987902 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:01:40.988042 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:40.988222 (+ 180us) server_negotiation.cc:299] Negotiation successful
0619 17:01:40.988361 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":85,"threads_started":1}
I20260619 17:01:40.990022 5023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:40.979868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:49871 (local address 127.20.231.190:41493)
0619 17:01:40.979977 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:40.979979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:40.980102 (+ 123us) server_negotiation.cc:407] Connection header received
0619 17:01:40.980889 (+ 787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:40.980890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:40.980904 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:40.980964 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:01:40.982037 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:40.982764 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:40.984097 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:40.984279 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:40.984368 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:40.986869 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:01:40.986871 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:01:40.986871 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:01:40.986878 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:01:40.988493 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:40.988983 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:40.988985 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:40.988985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:40.989053 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:40.989328 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:40.989329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:40.989329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:40.989455 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:01:40.989532 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:40.989854 (+ 322us) server_negotiation.cc:299] Negotiation successful
0619 17:01:40.989935 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 17:01:40.990245 5292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:40.979790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.131:49871)
0619 17:01:40.979975 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:40.980016 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 17:01:40.980766 (+ 750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:01:40.981040 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:01:40.981044 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:01:40.981658 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:01:40.981934 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:40.981940 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:40.982915 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:40.982916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:40.984008 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:40.984011 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:40.984385 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:40.984386 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:40.984505 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:40.985052 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:01:40.985060 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:01:40.986733 (+ 1673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:01:40.988574 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:01:40.988581 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:01:40.988583 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:01:40.988906 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:01:40.989121 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:01:40.989122 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:01:40.989122 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:01:40.989252 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:01:40.989533 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:01:40.989541 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:01:40.989697 (+ 156us) client_negotiation.cc:769] Sending connection context
0619 17:01:40.989827 (+ 130us) client_negotiation.cc:240] Negotiation successful
0619 17:01:40.989994 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":81,"threads_started":1}
I20260619 17:01:40.990702 5290 heartbeater.cc:347] Connected to a master server at 127.20.231.190:41493
I20260619 17:01:40.990808 5290 heartbeater.cc:464] Registering TS with master...
I20260619 17:01:40.991034 5290 heartbeater.cc:511] Master 127.20.231.190:41493 requested a full tablet report, sending...
I20260619 17:01:40.991384 4832 ts_manager.cc:194] Registered new tserver with Master: 168e7085a6724eda937626d694c09a10 (127.20.231.131:40067)
I20260619 17:01:40.991710 4832 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.131@KRBTEST.COM'} at 127.20.231.131:49871
I20260619 17:01:41.000494 21406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 17:01:41.003975 5023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:41.001099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58018 (local address 127.20.231.190:41493)
0619 17:01:41.001199 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:41.001201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:41.001211 (+ 10us) server_negotiation.cc:407] Connection header received
0619 17:01:41.001278 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:41.001279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:41.001318 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:41.001393 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 17:01:41.001596 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:41.002139 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:41.003198 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:41.003433 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:41.003502 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:41.003703 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 17:01:41.003754 (+ 51us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0619 17:01:41.003919 (+ 165us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58018: token expired
Metrics: {"server-negotiator.queue_time_us":71}
W20260619 17:01:41.004067 5023 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58018: token expired
W20260619 17:01:41.004237 4761 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.20.231.190:41493)
I20260619 17:01:41.011902 5023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:41.004666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58026 (local address 127.20.231.190:41493)
0619 17:01:41.004722 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:41.004724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:41.004807 (+ 83us) server_negotiation.cc:407] Connection header received
0619 17:01:41.004827 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:41.004828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:41.004836 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:41.004893 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:01:41.005403 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:41.005941 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:41.007142 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:41.007395 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:41.007457 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:41.008795 (+ 1338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:01:41.008796 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:01:41.008797 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:01:41.008805 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:01:41.010383 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:41.010774 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:41.010775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:41.010775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:41.010835 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:01:41.011210 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:01:41.011211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:01:41.011211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:01:41.011453 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:01:41.011547 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:41.011663 (+ 116us) server_negotiation.cc:299] Negotiation successful
0619 17:01:41.011736 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 17:01:41.013373 4832 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58026:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 17:01:41.015132 4832 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 17:01:41.017546 4954 tablet_service.cc:1467] Processing CreateTablet for tablet a98453a99ca84f93be185abbb273a043 (DEFAULT_TABLE table=test-table [id=a2bc9e172ecb4e09b753d50f5855c378]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 17:01:41.017781 4954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a98453a99ca84f93be185abbb273a043. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:01:41.019347 4912 tablet_bootstrap.cc:492] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2: Bootstrap starting.
I20260619 17:01:41.019697 4912 tablet_bootstrap.cc:654] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2: Neither blocks nor log segments found. Creating new log.
I20260619 17:01:41.020278 4912 tablet_bootstrap.cc:492] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2: No bootstrap required, opened a new log
I20260619 17:01:41.020337 4912 ts_tablet_manager.cc:1397] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260619 17:01:41.020444 4912 raft_consensus.cc:348] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e45f8a53c1a54a84943d3641364ebc67" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46227 } } peers { permanent_uuid: "168e7085a6724eda937626d694c09a10" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40067 } } peers { permanent_uuid: "ea83fb233f30420b8639ba14498653c2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 41923 } }
I20260619 17:01:41.020519 4912 raft_consensus.cc:374] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:01:41.020568 4912 raft_consensus.cc:729] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea83fb233f30420b8639ba14498653c2, State: Initialized, Role: FOLLOWER
I20260619 17:01:41.020615 4912 consensus_queue.cc:260] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e45f8a53c1a54a84943d3641364ebc67" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46227 } } peers { permanent_uuid: "168e7085a6724eda937626d694c09a10" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40067 } } peers { permanent_uuid: "ea83fb233f30420b8639ba14498653c2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 41923 } }
I20260619 17:01:41.020699 4912 ts_tablet_manager.cc:1428] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260619 17:01:41.021925 5296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:41.017379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:40067 (local address 127.0.0.1:43730)
0619 17:01:41.017775 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:41.017788 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 17:01:41.017853 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:01:41.018169 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:01:41.018170 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:01:41.018173 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:01:41.018242 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:41.018244 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:41.019600 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:41.019601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:41.020747 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:41.020750 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:41.021678 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:41.021679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:41.021741 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:41.021747 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 17:01:41.021799 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 17:01:41.021847 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374}
I20260619 17:01:41.022192 5297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:41.017574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43730 (local address 127.20.231.131:40067)
0619 17:01:41.017999 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:41.018004 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:41.018014 (+ 10us) server_negotiation.cc:407] Connection header received
0619 17:01:41.018058 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:41.018060 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:41.018073 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:41.018170 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:01:41.018303 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:41.019517 (+ 1214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:41.020911 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:41.021590 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:41.021681 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:41.021741 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:41.022035 (+ 294us) server_negotiation.cc:299] Negotiation successful
0619 17:01:41.022082 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":96,"threads_started":1}
I20260619 17:01:41.022795 5027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:41.017378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:46227 (local address 127.0.0.1:48016)
0619 17:01:41.018926 (+ 1548us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:41.018938 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 17:01:41.018984 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:01:41.019281 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:01:41.019282 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:01:41.019285 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:01:41.019338 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:41.019339 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:41.020644 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:41.020645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:41.021592 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:41.021594 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:41.022620 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:41.022621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:41.022667 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:41.022670 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 17:01:41.022714 (+ 44us) client_negotiation.cc:240] Negotiation successful
0619 17:01:41.022748 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1526,"mutex_wait_us":15,"thread_start_us":131,"threads_started":1}
I20260619 17:01:41.022867 5298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:41.017595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48016 (local address 127.20.231.130:46227)
0619 17:01:41.019116 (+ 1521us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:41.019121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:41.019130 (+ 9us) server_negotiation.cc:407] Connection header received
0619 17:01:41.019167 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:41.019169 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:41.019183 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:41.019267 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:01:41.019415 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:41.020451 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:41.021680 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:41.022530 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:41.022611 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:41.022664 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:41.022727 (+ 63us) server_negotiation.cc:299] Negotiation successful
0619 17:01:41.022766 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1481,"thread_start_us":100,"threads_started":1}
I20260619 17:01:41.023007 5224 tablet_service.cc:1467] Processing CreateTablet for tablet a98453a99ca84f93be185abbb273a043 (DEFAULT_TABLE table=test-table [id=a2bc9e172ecb4e09b753d50f5855c378]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 17:01:41.023362 5224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a98453a99ca84f93be185abbb273a043. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:01:41.023569 5092 tablet_service.cc:1467] Processing CreateTablet for tablet a98453a99ca84f93be185abbb273a043 (DEFAULT_TABLE table=test-table [id=a2bc9e172ecb4e09b753d50f5855c378]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 17:01:41.023918 5092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a98453a99ca84f93be185abbb273a043. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:01:41.025067 5026 raft_consensus.cc:482] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 17:01:41.025228 5026 raft_consensus.cc:504] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e45f8a53c1a54a84943d3641364ebc67" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46227 } } peers { permanent_uuid: "168e7085a6724eda937626d694c09a10" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40067 } } peers { permanent_uuid: "ea83fb233f30420b8639ba14498653c2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 41923 } }
I20260619 17:01:41.025988 5026 leader_election.cc:288] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e45f8a53c1a54a84943d3641364ebc67 (127.20.231.130:46227), 168e7085a6724eda937626d694c09a10 (127.20.231.131:40067)
I20260619 17:01:41.027909 5301 tablet_bootstrap.cc:492] T a98453a99ca84f93be185abbb273a043 P 168e7085a6724eda937626d694c09a10: Bootstrap starting.
I20260619 17:01:41.028360 5302 tablet_bootstrap.cc:492] T a98453a99ca84f93be185abbb273a043 P e45f8a53c1a54a84943d3641364ebc67: Bootstrap starting.
I20260619 17:01:41.028975 5301 tablet_bootstrap.cc:654] T a98453a99ca84f93be185abbb273a043 P 168e7085a6724eda937626d694c09a10: Neither blocks nor log segments found. Creating new log.
I20260619 17:01:41.028975 5302 tablet_bootstrap.cc:654] T a98453a99ca84f93be185abbb273a043 P e45f8a53c1a54a84943d3641364ebc67: Neither blocks nor log segments found. Creating new log.
I20260619 17:01:41.029647 5301 log.cc:824] T a98453a99ca84f93be185abbb273a043 P 168e7085a6724eda937626d694c09a10: Log is configured to *not* fsync() on all Append() calls
I20260619 17:01:41.029647 5302 log.cc:824] T a98453a99ca84f93be185abbb273a043 P e45f8a53c1a54a84943d3641364ebc67: Log is configured to *not* fsync() on all Append() calls
I20260619 17:01:41.031670 5022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:41.025906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:46227 (local address 127.20.231.129:52345)
0619 17:01:41.026898 (+ 992us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:41.026912 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 17:01:41.026961 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:01:41.027474 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:01:41.027476 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:01:41.027478 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:01:41.028074 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:41.028078 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:41.029270 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:41.029271 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:41.030540 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:41.030545 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:41.031356 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:41.031357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:41.031455 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:41.031459 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 17:01:41.031493 (+ 34us) client_negotiation.cc:240] Negotiation successful
0619 17:01:41.031541 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":961}
I20260619 17:01:41.032043 5298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:41.027154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:52345 (local address 127.20.231.130:46227)
0619 17:01:41.027223 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:41.027225 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:41.027242 (+ 17us) server_negotiation.cc:407] Connection header received
0619 17:01:41.027293 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:41.027293 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:41.027305 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:41.028192 (+ 887us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:01:41.028280 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:41.029143 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:41.030661 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:41.031242 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:41.031880 (+ 638us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:41.031892 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:41.031914 (+ 22us) server_negotiation.cc:299] Negotiation successful
0619 17:01:41.031960 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 17:01:41.033209 5302 tablet_bootstrap.cc:492] T a98453a99ca84f93be185abbb273a043 P e45f8a53c1a54a84943d3641364ebc67: No bootstrap required, opened a new log
I20260619 17:01:41.033286 5302 ts_tablet_manager.cc:1397] T a98453a99ca84f93be185abbb273a043 P e45f8a53c1a54a84943d3641364ebc67: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260619 17:01:41.032321 5112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a98453a99ca84f93be185abbb273a043" candidate_uuid: "ea83fb233f30420b8639ba14498653c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e45f8a53c1a54a84943d3641364ebc67" is_pre_election: true
I20260619 17:01:41.033916 5301 tablet_bootstrap.cc:492] T a98453a99ca84f93be185abbb273a043 P 168e7085a6724eda937626d694c09a10: No bootstrap required, opened a new log
W20260619 17:01:41.033954 4907 leader_election.cc:341] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e45f8a53c1a54a84943d3641364ebc67 (127.20.231.130:46227): Illegal state: must be running to vote when last-logged opid is not known
I20260619 17:01:41.034107 5301 ts_tablet_manager.cc:1397] T a98453a99ca84f93be185abbb273a043 P 168e7085a6724eda937626d694c09a10: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260619 17:01:41.034680 5302 raft_consensus.cc:348] T a98453a99ca84f93be185abbb273a043 P e45f8a53c1a54a84943d3641364ebc67 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e45f8a53c1a54a84943d3641364ebc67" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46227 } } peers { permanent_uuid: "168e7085a6724eda937626d694c09a10" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40067 } } peers { permanent_uuid: "ea83fb233f30420b8639ba14498653c2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 41923 } }
I20260619 17:01:41.034839 5302 raft_consensus.cc:374] T a98453a99ca84f93be185abbb273a043 P e45f8a53c1a54a84943d3641364ebc67 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:01:41.034925 5302 raft_consensus.cc:729] T a98453a99ca84f93be185abbb273a043 P e45f8a53c1a54a84943d3641364ebc67 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e45f8a53c1a54a84943d3641364ebc67, State: Initialized, Role: FOLLOWER
I20260619 17:01:41.035053 5302 consensus_queue.cc:260] T a98453a99ca84f93be185abbb273a043 P e45f8a53c1a54a84943d3641364ebc67 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e45f8a53c1a54a84943d3641364ebc67" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46227 } } peers { permanent_uuid: "168e7085a6724eda937626d694c09a10" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40067 } } peers { permanent_uuid: "ea83fb233f30420b8639ba14498653c2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 41923 } }
I20260619 17:01:41.035334 5302 ts_tablet_manager.cc:1428] T a98453a99ca84f93be185abbb273a043 P e45f8a53c1a54a84943d3641364ebc67: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 17:01:41.035375 5158 heartbeater.cc:503] Master 127.20.231.190:41493 was elected leader, sending a full tablet report...
I20260619 17:01:41.035792 5297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:41.027124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:43555 (local address 127.20.231.131:40067)
0619 17:01:41.027998 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:41.028001 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:41.030028 (+ 2027us) server_negotiation.cc:407] Connection header received
0619 17:01:41.030140 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:41.030141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:41.030150 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:41.030217 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:01:41.030375 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:41.031266 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:41.032724 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:41.033275 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:41.033363 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:41.033374 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:41.035662 (+ 2288us) server_negotiation.cc:299] Negotiation successful
0619 17:01:41.035718 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":852}
I20260619 17:01:41.035802 5300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:41.026187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:40067 (local address 127.20.231.129:43555)
0619 17:01:41.029959 (+ 3772us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:41.029972 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 17:01:41.030053 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:01:41.030223 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:01:41.030224 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:01:41.030226 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:01:41.030302 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:41.030304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:41.031378 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:41.031379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:41.032637 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:41.032641 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:41.035447 (+ 2806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:41.035449 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:41.035536 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:41.035541 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 17:01:41.035647 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 17:01:41.035700 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3746,"thread_start_us":163,"threads_started":1}
I20260619 17:01:41.036104 5244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a98453a99ca84f93be185abbb273a043" candidate_uuid: "ea83fb233f30420b8639ba14498653c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "168e7085a6724eda937626d694c09a10" is_pre_election: true
W20260619 17:01:41.036471 4908 leader_election.cc:341] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 168e7085a6724eda937626d694c09a10 (127.20.231.131:40067): Illegal state: must be running to vote when last-logged opid is not known
I20260619 17:01:41.036399 5301 raft_consensus.cc:348] T a98453a99ca84f93be185abbb273a043 P 168e7085a6724eda937626d694c09a10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e45f8a53c1a54a84943d3641364ebc67" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46227 } } peers { permanent_uuid: "168e7085a6724eda937626d694c09a10" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40067 } } peers { permanent_uuid: "ea83fb233f30420b8639ba14498653c2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 41923 } }
I20260619 17:01:41.036530 5301 raft_consensus.cc:374] T a98453a99ca84f93be185abbb273a043 P 168e7085a6724eda937626d694c09a10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:01:41.036525 4908 leader_election.cc:302] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ea83fb233f30420b8639ba14498653c2; no voters: 168e7085a6724eda937626d694c09a10, e45f8a53c1a54a84943d3641364ebc67
I20260619 17:01:41.036562 5301 raft_consensus.cc:729] T a98453a99ca84f93be185abbb273a043 P 168e7085a6724eda937626d694c09a10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 168e7085a6724eda937626d694c09a10, State: Initialized, Role: FOLLOWER
I20260619 17:01:41.036700 5026 raft_consensus.cc:2726] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 17:01:41.036705 5301 consensus_queue.cc:260] T a98453a99ca84f93be185abbb273a043 P 168e7085a6724eda937626d694c09a10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e45f8a53c1a54a84943d3641364ebc67" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46227 } } peers { permanent_uuid: "168e7085a6724eda937626d694c09a10" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40067 } } peers { permanent_uuid: "ea83fb233f30420b8639ba14498653c2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 41923 } }
I20260619 17:01:41.036979 5290 heartbeater.cc:503] Master 127.20.231.190:41493 was elected leader, sending a full tablet report...
I20260619 17:01:41.036979 5301 ts_tablet_manager.cc:1428] T a98453a99ca84f93be185abbb273a043 P 168e7085a6724eda937626d694c09a10: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260619 17:01:41.094129 5159 tablet.cc:2369] T a98453a99ca84f93be185abbb273a043 P e45f8a53c1a54a84943d3641364ebc67: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 17:01:41.195546 5021 tablet.cc:2369] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 17:01:41.229772 5291 tablet.cc:2369] T a98453a99ca84f93be185abbb273a043 P 168e7085a6724eda937626d694c09a10: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 17:01:41.282203 5306 raft_consensus.cc:482] T a98453a99ca84f93be185abbb273a043 P 168e7085a6724eda937626d694c09a10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 17:01:41.282380 5306 raft_consensus.cc:504] T a98453a99ca84f93be185abbb273a043 P 168e7085a6724eda937626d694c09a10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e45f8a53c1a54a84943d3641364ebc67" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46227 } } peers { permanent_uuid: "168e7085a6724eda937626d694c09a10" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40067 } } peers { permanent_uuid: "ea83fb233f30420b8639ba14498653c2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 41923 } }
I20260619 17:01:41.282691 5306 leader_election.cc:288] T a98453a99ca84f93be185abbb273a043 P 168e7085a6724eda937626d694c09a10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e45f8a53c1a54a84943d3641364ebc67 (127.20.231.130:46227), ea83fb233f30420b8639ba14498653c2 (127.20.231.129:41923)
I20260619 17:01:41.287020 5298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:41.283074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:60085 (local address 127.20.231.130:46227)
0619 17:01:41.283132 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:41.283135 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:41.283150 (+ 15us) server_negotiation.cc:407] Connection header received
0619 17:01:41.283206 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:41.283207 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:41.283219 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:41.283290 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:01:41.283472 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:41.284336 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:41.285937 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:41.286584 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:41.286670 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:41.286680 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:41.286890 (+ 210us) server_negotiation.cc:299] Negotiation successful
0619 17:01:41.286947 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 17:01:41.287006 5292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:41.282968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:46227 (local address 127.20.231.131:60085)
0619 17:01:41.283040 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:41.283055 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 17:01:41.283126 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:01:41.283299 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:01:41.283300 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:01:41.283302 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:01:41.283385 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:41.283387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:41.284451 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:41.284453 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:41.285795 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:41.285799 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:41.286679 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:41.286680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:41.286766 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:41.286774 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 17:01:41.286833 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 17:01:41.286911 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":59}
I20260619 17:01:41.287083 5308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:41.283076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:38003 (local address 127.20.231.129:41923)
0619 17:01:41.283227 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:41.283230 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:41.283334 (+ 104us) server_negotiation.cc:407] Connection header received
0619 17:01:41.283366 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:41.283367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:41.283381 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:41.283484 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:01:41.283630 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:41.284468 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:41.285954 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:41.286584 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:41.286671 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:41.286680 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:41.286950 (+ 270us) server_negotiation.cc:299] Negotiation successful
0619 17:01:41.286999 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":64,"threads_started":1}
I20260619 17:01:41.287034 5307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:41.282970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:41923 (local address 127.20.231.131:38003)
0619 17:01:41.283229 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:41.283238 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 17:01:41.283311 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:01:41.283504 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:01:41.283505 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:01:41.283506 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:01:41.283552 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:41.283553 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:41.284559 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:41.284560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:41.285799 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:01:41.285800 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:01:41.286680 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:01:41.286680 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:01:41.286766 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:41.286774 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 17:01:41.286932 (+ 158us) client_negotiation.cc:240] Negotiation successful
0619 17:01:41.286964 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":60,"thread_start_us":111,"threads_started":1}
I20260619 17:01:41.287366 5112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a98453a99ca84f93be185abbb273a043" candidate_uuid: "168e7085a6724eda937626d694c09a10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e45f8a53c1a54a84943d3641364ebc67" is_pre_election: true
I20260619 17:01:41.287570 5112 raft_consensus.cc:2454] T a98453a99ca84f93be185abbb273a043 P e45f8a53c1a54a84943d3641364ebc67 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 168e7085a6724eda937626d694c09a10 in term 0.
I20260619 17:01:41.287599 4974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a98453a99ca84f93be185abbb273a043" candidate_uuid: "168e7085a6724eda937626d694c09a10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea83fb233f30420b8639ba14498653c2" is_pre_election: true
I20260619 17:01:41.287777 4974 raft_consensus.cc:2454] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 168e7085a6724eda937626d694c09a10 in term 0.
I20260619 17:01:41.287811 5178 leader_election.cc:302] T a98453a99ca84f93be185abbb273a043 P 168e7085a6724eda937626d694c09a10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 168e7085a6724eda937626d694c09a10, e45f8a53c1a54a84943d3641364ebc67; no voters:
I20260619 17:01:41.287966 5306 raft_consensus.cc:2781] T a98453a99ca84f93be185abbb273a043 P 168e7085a6724eda937626d694c09a10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 17:01:41.288053 5306 raft_consensus.cc:482] T a98453a99ca84f93be185abbb273a043 P 168e7085a6724eda937626d694c09a10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 17:01:41.288072 5306 raft_consensus.cc:3037] T a98453a99ca84f93be185abbb273a043 P 168e7085a6724eda937626d694c09a10 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:01:41.288857 5306 raft_consensus.cc:504] T a98453a99ca84f93be185abbb273a043 P 168e7085a6724eda937626d694c09a10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e45f8a53c1a54a84943d3641364ebc67" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46227 } } peers { permanent_uuid: "168e7085a6724eda937626d694c09a10" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40067 } } peers { permanent_uuid: "ea83fb233f30420b8639ba14498653c2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 41923 } }
I20260619 17:01:41.289028 5306 leader_election.cc:288] T a98453a99ca84f93be185abbb273a043 P 168e7085a6724eda937626d694c09a10 [CANDIDATE]: Term 1 election: Requested vote from peers e45f8a53c1a54a84943d3641364ebc67 (127.20.231.130:46227), ea83fb233f30420b8639ba14498653c2 (127.20.231.129:41923)
I20260619 17:01:41.289194 5112 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a98453a99ca84f93be185abbb273a043" candidate_uuid: "168e7085a6724eda937626d694c09a10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e45f8a53c1a54a84943d3641364ebc67"
I20260619 17:01:41.289217 4974 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a98453a99ca84f93be185abbb273a043" candidate_uuid: "168e7085a6724eda937626d694c09a10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ea83fb233f30420b8639ba14498653c2"
I20260619 17:01:41.289330 4974 raft_consensus.cc:3037] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:01:41.289302 5112 raft_consensus.cc:3037] T a98453a99ca84f93be185abbb273a043 P e45f8a53c1a54a84943d3641364ebc67 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:01:41.289992 5112 raft_consensus.cc:2454] T a98453a99ca84f93be185abbb273a043 P e45f8a53c1a54a84943d3641364ebc67 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 168e7085a6724eda937626d694c09a10 in term 1.
I20260619 17:01:41.289996 4974 raft_consensus.cc:2454] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 168e7085a6724eda937626d694c09a10 in term 1.
I20260619 17:01:41.290170 5178 leader_election.cc:302] T a98453a99ca84f93be185abbb273a043 P 168e7085a6724eda937626d694c09a10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 168e7085a6724eda937626d694c09a10, e45f8a53c1a54a84943d3641364ebc67; no voters:
I20260619 17:01:41.290287 5306 raft_consensus.cc:2781] T a98453a99ca84f93be185abbb273a043 P 168e7085a6724eda937626d694c09a10 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:01:41.290477 5306 raft_consensus.cc:686] T a98453a99ca84f93be185abbb273a043 P 168e7085a6724eda937626d694c09a10 [term 1 LEADER]: Becoming Leader. State: Replica: 168e7085a6724eda937626d694c09a10, State: Running, Role: LEADER
I20260619 17:01:41.290591 5306 consensus_queue.cc:237] T a98453a99ca84f93be185abbb273a043 P 168e7085a6724eda937626d694c09a10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e45f8a53c1a54a84943d3641364ebc67" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46227 } } peers { permanent_uuid: "168e7085a6724eda937626d694c09a10" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40067 } } peers { permanent_uuid: "ea83fb233f30420b8639ba14498653c2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 41923 } }
I20260619 17:01:41.291497 4832 catalog_manager.cc:5495] T a98453a99ca84f93be185abbb273a043 P 168e7085a6724eda937626d694c09a10 reported cstate change: term changed from 0 to 1, leader changed from <none> to 168e7085a6724eda937626d694c09a10 (127.20.231.131). New cstate: current_term: 1 leader_uuid: "168e7085a6724eda937626d694c09a10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e45f8a53c1a54a84943d3641364ebc67" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46227 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "168e7085a6724eda937626d694c09a10" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40067 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ea83fb233f30420b8639ba14498653c2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 41923 } health_report { overall_health: UNKNOWN } } }
I20260619 17:01:41.317427 5297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:41.314976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43734 (local address 127.20.231.131:40067)
0619 17:01:41.315061 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0619 17:01:41.315063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:01:41.315072 (+ 9us) server_negotiation.cc:407] Connection header received
0619 17:01:41.315112 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:01:41.315113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:01:41.315135 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:01:41.315204 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 17:01:41.315322 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:41.315848 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:41.316574 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:01:41.316780 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:01:41.316914 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:01:41.317031 (+ 117us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 17:01:41.317141 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 17:01:41.317226 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 17:01:41.317239 (+ 13us) server_negotiation.cc:299] Negotiation successful
0619 17:01:41.317259 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 17:01:41.319428 5112 raft_consensus.cc:1261] T a98453a99ca84f93be185abbb273a043 P e45f8a53c1a54a84943d3641364ebc67 [term 1 FOLLOWER]: Refusing update from remote peer 168e7085a6724eda937626d694c09a10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 17:01:41.319448 4974 raft_consensus.cc:1261] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [term 1 FOLLOWER]: Refusing update from remote peer 168e7085a6724eda937626d694c09a10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 17:01:41.319810 5306 consensus_queue.cc:1035] T a98453a99ca84f93be185abbb273a043 P 168e7085a6724eda937626d694c09a10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e45f8a53c1a54a84943d3641364ebc67" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46227 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 17:01:41.320093 5309 consensus_queue.cc:1035] T a98453a99ca84f93be185abbb273a043 P 168e7085a6724eda937626d694c09a10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea83fb233f30420b8639ba14498653c2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 41923 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 17:01:41.322204 5314 mvcc.cc:205] Tried to move back new op lower bound from 7298615301401161728 to 7298615301286899712. Current Snapshot: MvccSnapshot[applied={T|T < 7298615301401161728}]
I20260619 17:01:41.322196 5312 mvcc.cc:205] Tried to move back new op lower bound from 7298615301401161728 to 7298615301286899712. Current Snapshot: MvccSnapshot[applied={T|T < 7298615301401161728}]
I20260619 17:01:41.325295 4831 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 17:01:41.325399 4831 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 17:01:41.326205 4831 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27: Sending DeleteTablet for 3 replicas of tablet a98453a99ca84f93be185abbb273a043
I20260619 17:01:41.326570 5092 tablet_service.cc:1514] Processing DeleteTablet for tablet a98453a99ca84f93be185abbb273a043 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:01:41 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:48016
I20260619 17:01:41.326646 4954 tablet_service.cc:1514] Processing DeleteTablet for tablet a98453a99ca84f93be185abbb273a043 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:01:41 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:44444
I20260619 17:01:41.326622 5224 tablet_service.cc:1514] Processing DeleteTablet for tablet a98453a99ca84f93be185abbb273a043 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:01:41 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:43730
I20260619 17:01:41.326833 5323 tablet_replica.cc:330] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2: stopping tablet replica
I20260619 17:01:41.326897 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 4870
I20260619 17:01:41.326920 5323 raft_consensus.cc:2229] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 17:01:41.326967 5323 raft_consensus.cc:2258] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 17:01:41.327122 5322 tablet_replica.cc:330] T a98453a99ca84f93be185abbb273a043 P e45f8a53c1a54a84943d3641364ebc67: stopping tablet replica
I20260619 17:01:41.327265 5322 raft_consensus.cc:2229] T a98453a99ca84f93be185abbb273a043 P e45f8a53c1a54a84943d3641364ebc67 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 17:01:41.327531 5322 raft_consensus.cc:2258] T a98453a99ca84f93be185abbb273a043 P e45f8a53c1a54a84943d3641364ebc67 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 17:01:41.328063 5322 ts_tablet_manager.cc:1905] T a98453a99ca84f93be185abbb273a043 P e45f8a53c1a54a84943d3641364ebc67: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 17:01:41.330195 5322 ts_tablet_manager.cc:1918] T a98453a99ca84f93be185abbb273a043 P e45f8a53c1a54a84943d3641364ebc67: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 17:01:41.330291 5322 log.cc:1196] T a98453a99ca84f93be185abbb273a043 P e45f8a53c1a54a84943d3641364ebc67: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/wal/wals/a98453a99ca84f93be185abbb273a043
I20260619 17:01:41.330588 5322 ts_tablet_manager.cc:1939] T a98453a99ca84f93be185abbb273a043 P e45f8a53c1a54a84943d3641364ebc67: Deleting consensus metadata
I20260619 17:01:41.331099 4817 catalog_manager.cc:4841] TS e45f8a53c1a54a84943d3641364ebc67 (127.20.231.130:46227): tablet a98453a99ca84f93be185abbb273a043 (table test-table [id=a2bc9e172ecb4e09b753d50f5855c378]) successfully deleted
W20260619 17:01:41.332911 4816 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.231.129:41923 (error 108)
I20260619 17:01:41.333396 5027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:01:41.333258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:41923 (local address 127.0.0.1:44460)
0619 17:01:41.333306 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 17:01:41.333355 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.129:41923: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260619 17:01:41.333468 5324 tablet_replica.cc:330] T a98453a99ca84f93be185abbb273a043 P 168e7085a6724eda937626d694c09a10: stopping tablet replica
I20260619 17:01:41.333561 5324 raft_consensus.cc:2229] T a98453a99ca84f93be185abbb273a043 P 168e7085a6724eda937626d694c09a10 [term 1 LEADER]: Raft consensus shutting down.
I20260619 17:01:41.333693 5324 raft_consensus.cc:2258] T a98453a99ca84f93be185abbb273a043 P 168e7085a6724eda937626d694c09a10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 17:01:41.333911 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 5025
W20260619 17:01:41.333583 4816 catalog_manager.cc:4584] TS ea83fb233f30420b8639ba14498653c2 (127.20.231.129:41923): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a98453a99ca84f93be185abbb273a043: Network error: Client connection negotiation failed: client connection to 127.20.231.129:41923: connect: Connection refused (error 111)
I20260619 17:01:41.336336 5324 ts_tablet_manager.cc:1905] T a98453a99ca84f93be185abbb273a043 P 168e7085a6724eda937626d694c09a10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 17:01:41.338028 5324 ts_tablet_manager.cc:1918] T a98453a99ca84f93be185abbb273a043 P 168e7085a6724eda937626d694c09a10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 17:01:41.338121 5324 log.cc:1196] T a98453a99ca84f93be185abbb273a043 P 168e7085a6724eda937626d694c09a10: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/wal/wals/a98453a99ca84f93be185abbb273a043
I20260619 17:01:41.338394 5324 ts_tablet_manager.cc:1939] T a98453a99ca84f93be185abbb273a043 P 168e7085a6724eda937626d694c09a10: Deleting consensus metadata
I20260619 17:01:41.338874 4818 catalog_manager.cc:4841] TS 168e7085a6724eda937626d694c09a10 (127.20.231.131:40067): tablet a98453a99ca84f93be185abbb273a043 (table test-table [id=a2bc9e172ecb4e09b753d50f5855c378]) successfully deleted
I20260619 17:01:41.339957 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 5161
I20260619 17:01:41.344668 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 4799
I20260619 17:02:01.350265 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:41493
--webserver_interface=127.20.231.190
--webserver_port=33955
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:43231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:41493
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:02:01.434813 5326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:01.435132 5326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:01.435173 5326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:01.436421 5326 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 17:02:01.436512 5326 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 17:02:01.436534 5326 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 17:02:01.436555 5326 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:02:01.436584 5326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:01.436614 5326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:01.436632 5326 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:02:01.436654 5326 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:02:01.438236 5326 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:43231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:41493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:41493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=33955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:01.438556 5326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:01.438812 5326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:01.444023 5331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:01.444056 5332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:01.444017 5334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:01.444139 5326 server_base.cc:1034] running on GCE node
I20260619 17:02:01.444504 5326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:01.444871 5326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:01.446070 5326 hybrid_clock.cc:648] HybridClock initialized: now 1781888521446046 us; error 40 us; skew 500 ppm
Jun 19 17:02:01 dist-test-slave-vnbj krb5kdc[4250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888521, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:01.455855 5326 init.cc:377] Logged in from keytab as kudu/127.20.231.190@KRBTEST.COM (short username kudu)
I20260619 17:02:01.456506 5326 webserver.cc:460] Webserver started at http://127.20.231.190:33955/ using document root <none> and password file <none>
I20260619 17:02:01.456785 5326 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:01.456866 5326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:01.458611 5326 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260619 17:02:01.459650 5341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:01.460017 5326 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 17:02:01.460146 5326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "e32dbf7acea04f4a83288bb09da26c27"
format_stamp: "Formatted at 2026-06-19 17:01:19 on dist-test-slave-vnbj"
server_key: "918405ff9790103545cff23661c5ccd2"
server_key_iv: "aa71d0a836a4aa888a18580ee37a45eb"
server_key_version: "encryptionkey@0"
I20260619 17:02:01.460598 5326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:01.478032 5326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:01.481140 5326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:01.481369 5326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:01.486277 5326 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:41493
I20260619 17:02:01.486338 5393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:41493 every 8 connection(s)
I20260619 17:02:01.486796 5326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 17:02:01.488400 5394 sys_catalog.cc:263] Verifying existing consensus state
I20260619 17:02:01.489046 5394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27: Bootstrap starting.
I20260619 17:02:01.496642 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 5326
I20260619 17:02:01.497007 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.129:41923
--local_ip_for_outbound_sockets=127.20.231.129
--tserver_master_addrs=127.20.231.190:41493
--webserver_port=43213
--webserver_interface=127.20.231.129
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:43231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 17:02:01.498092 5394 log.cc:824] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:01.500628 5394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 17:02:01.501003 5394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27: Bootstrap complete.
I20260619 17:02:01.503484 5394 raft_consensus.cc:348] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e32dbf7acea04f4a83288bb09da26c27" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41493 } }
I20260619 17:02:01.503768 5394 raft_consensus.cc:729] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e32dbf7acea04f4a83288bb09da26c27, State: Initialized, Role: FOLLOWER
I20260619 17:02:01.503983 5394 consensus_queue.cc:260] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e32dbf7acea04f4a83288bb09da26c27" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41493 } }
I20260619 17:02:01.504073 5394 raft_consensus.cc:388] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 17:02:01.504128 5394 raft_consensus.cc:482] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 17:02:01.504184 5394 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [term 2 FOLLOWER]: Advancing to term 3
I20260619 17:02:01.505293 5394 raft_consensus.cc:504] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e32dbf7acea04f4a83288bb09da26c27" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41493 } }
I20260619 17:02:01.505440 5394 leader_election.cc:302] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e32dbf7acea04f4a83288bb09da26c27; no voters:
I20260619 17:02:01.505677 5394 leader_election.cc:288] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 17:02:01.505714 5398 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 17:02:01.505829 5398 raft_consensus.cc:686] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [term 3 LEADER]: Becoming Leader. State: Replica: e32dbf7acea04f4a83288bb09da26c27, State: Running, Role: LEADER
I20260619 17:02:01.505924 5398 consensus_queue.cc:237] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e32dbf7acea04f4a83288bb09da26c27" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41493 } }
I20260619 17:02:01.505982 5394 sys_catalog.cc:564] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:02:01.506196 5398 sys_catalog.cc:455] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e32dbf7acea04f4a83288bb09da26c27. Latest consensus state: current_term: 3 leader_uuid: "e32dbf7acea04f4a83288bb09da26c27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e32dbf7acea04f4a83288bb09da26c27" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41493 } } }
I20260619 17:02:01.506265 5398 sys_catalog.cc:458] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:01.506413 5399 sys_catalog.cc:455] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "e32dbf7acea04f4a83288bb09da26c27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e32dbf7acea04f4a83288bb09da26c27" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41493 } } }
I20260619 17:02:01.506490 5399 sys_catalog.cc:458] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27 [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:01.506618 5402 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 17:02:01.507609 5402 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 17:02:01.507722 5402 catalog_manager.cc:1259] Loaded cluster ID: 9e590b6b1b1c4b0faa086a41bc51f959
I20260619 17:02:01.507754 5402 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 17:02:01.508708 5402 catalog_manager.cc:1504] Loading token signing keys...
I20260619 17:02:01.508991 5402 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27: Loaded TSK: 0
I20260619 17:02:01.509061 5402 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e32dbf7acea04f4a83288bb09da26c27: Loaded TSK: 1
F20260619 17:02:01.509300 5402 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f234f2af9ad google::LogMessage::Fail() at ??:0
@ 0x7f234f2b288a google::LogMessage::SendToLog() at ??:0
@ 0x7f234f2af4de google::LogMessage::Flush() at ??:0
@ 0x7f234f2b0a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f2351a1c104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f2351a19d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f235720d1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f235720d48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f235720d594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f23571d3c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f23571ddad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f235117d0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f2351172954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f2351fd66db start_thread at ??:0
@ 0x7f234ddad61f clone at ??:0
W20260619 17:02:01.627908 5396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:01.628150 5396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:01.628182 5396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:01.629709 5396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:01.629773 5396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:01.629808 5396 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.129
I20260619 17:02:01.632444 5396 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:43231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.129:41923
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.231.129
--webserver_port=43213
--tserver_master_addrs=127.20.231.190:41493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.129
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:01.632808 5396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:01.633123 5396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:01.641309 5438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:01.641635 5436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:01.641856 5435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:01.643527 5396 server_base.cc:1034] running on GCE node
I20260619 17:02:01.643821 5396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:01.644153 5396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:01.645412 5396 hybrid_clock.cc:648] HybridClock initialized: now 1781888521645376 us; error 55 us; skew 500 ppm
Jun 19 17:02:01 dist-test-slave-vnbj krb5kdc[4250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888521, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:01.648643 5396 init.cc:377] Logged in from keytab as kudu/127.20.231.129@KRBTEST.COM (short username kudu)
I20260619 17:02:01.649256 5396 webserver.cc:460] Webserver started at http://127.20.231.129:43213/ using document root <none> and password file <none>
I20260619 17:02:01.649592 5396 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:01.649680 5396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:01.651522 5396 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 17:02:01.652561 5445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:01.652879 5396 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:01.652990 5396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/wal
uuid: "ea83fb233f30420b8639ba14498653c2"
format_stamp: "Formatted at 2026-06-19 17:01:19 on dist-test-slave-vnbj"
server_key: "98d201fecad496ed9a77e9331febb554"
server_key_iv: "c906756b4af81417da67a7b2598c1f28"
server_key_version: "encryptionkey@0"
I20260619 17:02:01.653506 5396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:01.667217 5396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:01.669812 5396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:01.669971 5396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:01.670257 5396 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:02:01.670727 5452 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 17:02:01.672823 5396 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 17:02:01.672910 5396 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260619 17:02:01.672960 5396 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 17:02:01.673765 5396 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 17:02:01.673825 5396 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260619 17:02:01.673916 5452 tablet_bootstrap.cc:492] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2: Bootstrap starting.
I20260619 17:02:01.682029 5396 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.129:41923
I20260619 17:02:01.682168 5559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.129:41923 every 8 connection(s)
I20260619 17:02:01.682483 5396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
I20260619 17:02:01.683883 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 5396
I20260619 17:02:01.684202 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.130:46227
--local_ip_for_outbound_sockets=127.20.231.130
--tserver_master_addrs=127.20.231.190:41493
--webserver_port=42209
--webserver_interface=127.20.231.130
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:43231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 17:02:01.686296 5452 log.cc:824] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:01.689560 5452 tablet_bootstrap.cc:492] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 17:02:01.689946 5452 tablet_bootstrap.cc:492] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2: Bootstrap complete.
I20260619 17:02:01.691293 5452 ts_tablet_manager.cc:1397] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.007s
I20260619 17:02:01.693289 5452 raft_consensus.cc:348] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e45f8a53c1a54a84943d3641364ebc67" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46227 } } peers { permanent_uuid: "168e7085a6724eda937626d694c09a10" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40067 } } peers { permanent_uuid: "ea83fb233f30420b8639ba14498653c2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 41923 } }
I20260619 17:02:01.693753 5452 raft_consensus.cc:729] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea83fb233f30420b8639ba14498653c2, State: Initialized, Role: FOLLOWER
I20260619 17:02:01.693933 5452 consensus_queue.cc:260] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e45f8a53c1a54a84943d3641364ebc67" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46227 } } peers { permanent_uuid: "168e7085a6724eda937626d694c09a10" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40067 } } peers { permanent_uuid: "ea83fb233f30420b8639ba14498653c2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 41923 } }
I20260619 17:02:01.694300 5452 ts_tablet_manager.cc:1428] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
W20260619 17:02:01.749277 5562 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 17:02:01.683300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.129:42055)
0619 17:02:01.683490 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:01.683536 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:01.684559 (+ 1023us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:01.749043 (+ 64484us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":112,"threads_started":1}
W20260619 17:02:01.749605 5560 heartbeater.cc:644] Failed to heartbeat to 127.20.231.190:41493 (0 consecutive failures): Network error: Failed to ping master at 127.20.231.190:41493: Client connection negotiation failed: client connection to 127.20.231.190:41493: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 17:02:01.750090 5562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:01.749928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.129:51729)
0619 17:02:01.750014 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:01.750065 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260619 17:02:01.750440 5562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:01.750336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.129:51231)
0619 17:02:01.750389 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:01.750416 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 17:02:01.750555 5560 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 17:02:01.778007 5563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:01.778330 5563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:01.778381 5563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:01.779840 5563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:01.779939 5563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:01.779991 5563 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.130
I20260619 17:02:01.781720 5563 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:43231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.130:46227
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.231.130
--webserver_port=42209
--tserver_master_addrs=127.20.231.190:41493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.130
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:01.782063 5563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:01.782337 5563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:01.789000 5570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:01.789153 5573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:01.789255 5571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:01.789467 5563 server_base.cc:1034] running on GCE node
I20260619 17:02:01.789714 5563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:01.790153 5563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:01.791757 5563 hybrid_clock.cc:648] HybridClock initialized: now 1781888521791679 us; error 97 us; skew 500 ppm
Jun 19 17:02:01 dist-test-slave-vnbj krb5kdc[4250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888521, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:01.796547 5563 init.cc:377] Logged in from keytab as kudu/127.20.231.130@KRBTEST.COM (short username kudu)
I20260619 17:02:01.797310 5563 webserver.cc:460] Webserver started at http://127.20.231.130:42209/ using document root <none> and password file <none>
I20260619 17:02:01.797622 5563 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:01.797731 5563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:01.799860 5563 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 17:02:01.800776 5580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:01.801184 5563 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 17:02:01.801375 5563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/wal
uuid: "e45f8a53c1a54a84943d3641364ebc67"
format_stamp: "Formatted at 2026-06-19 17:01:20 on dist-test-slave-vnbj"
server_key: "34d862066aba9d978dc9fe26115eabf3"
server_key_iv: "51eb8d12855df27c15131e960fa1e49a"
server_key_version: "encryptionkey@0"
I20260619 17:02:01.801898 5563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:01.841521 5563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:01.844166 5563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:01.844375 5563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:01.844655 5563 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:02:01.845027 5563 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:02:01.845103 5563 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:01.845149 5563 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:02:01.845170 5563 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:01.851912 5563 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.130:46227
I20260619 17:02:01.851992 5693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.130:46227 every 8 connection(s)
I20260619 17:02:01.852417 5563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
I20260619 17:02:01.853878 5696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:01.853217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.130:39431)
0619 17:02:01.853404 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:01.853486 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":73,"threads_started":1}
W20260619 17:02:01.854180 5694 heartbeater.cc:644] Failed to heartbeat to 127.20.231.190:41493 (0 consecutive failures): Network error: Failed to ping master at 127.20.231.190:41493: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
I20260619 17:02:01.854564 5696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:01.854409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.130:52847)
0619 17:02:01.854506 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:01.854541 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260619 17:02:01.855132 5696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:01.855019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.130:60215)
0619 17:02:01.855074 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:01.855108 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 17:02:01.855285 5694 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 17:02:01.861479 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 5563
I20260619 17:02:01.861901 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.131:40067
--local_ip_for_outbound_sockets=127.20.231.131
--tserver_master_addrs=127.20.231.190:41493
--webserver_port=37707
--webserver_interface=127.20.231.131
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:43231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:02:01.948583 5697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:01.948949 5697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:01.949003 5697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:01.950614 5697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:01.950728 5697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:01.950795 5697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.131
I20260619 17:02:01.952785 5697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:43231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.131:40067
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.231.131
--webserver_port=37707
--tserver_master_addrs=127.20.231.190:41493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.131
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:01.953171 5697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:01.953480 5697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:01.960381 5705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:01.960343 5702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:01.960357 5703 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:01.960574 5697 server_base.cc:1034] running on GCE node
I20260619 17:02:01.960863 5697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:01.961210 5697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:01.962482 5697 hybrid_clock.cc:648] HybridClock initialized: now 1781888521962479 us; error 76 us; skew 500 ppm
Jun 19 17:02:01 dist-test-slave-vnbj krb5kdc[4250](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888521, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:01.965556 5697 init.cc:377] Logged in from keytab as kudu/127.20.231.131@KRBTEST.COM (short username kudu)
I20260619 17:02:01.966063 5697 webserver.cc:460] Webserver started at http://127.20.231.131:37707/ using document root <none> and password file <none>
I20260619 17:02:01.966358 5697 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:01.966451 5697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:01.968151 5697 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:01.969007 5712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:01.969343 5697 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:01.969480 5697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/wal
uuid: "168e7085a6724eda937626d694c09a10"
format_stamp: "Formatted at 2026-06-19 17:01:20 on dist-test-slave-vnbj"
server_key: "84c4a271b887166f0a1169318be2ec48"
server_key_iv: "da296dbec863eec43149ecf3353e0ff0"
server_key_version: "encryptionkey@0"
I20260619 17:02:01.969895 5697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:01.988209 5697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:01.990934 5697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:01.991117 5697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:01.991398 5697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:02:01.991794 5697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:02:01.991873 5697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:01.991915 5697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:02:01.991935 5697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:01.998288 5697 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.131:40067
I20260619 17:02:01.998374 5825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.131:40067 every 8 connection(s)
I20260619 17:02:01.998891 5697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
I20260619 17:02:02.000190 5828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:01.999665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.131:36351)
0619 17:02:01.999967 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:02.000045 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":78,"threads_started":1}
W20260619 17:02:02.000670 5826 heartbeater.cc:644] Failed to heartbeat to 127.20.231.190:41493 (0 consecutive failures): Network error: Failed to ping master at 127.20.231.190:41493: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
I20260619 17:02:02.001060 5828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:02.000961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.131:48133)
0619 17:02:02.001014 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:02.001041 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 17:02:02.001364 5828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:02.001275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.131:37011)
0619 17:02:02.001320 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:02.001346 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260619 17:02:02.001531 5826 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 17:02:02.008494 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 5697
I20260619 17:02:02.751340 5830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:02.751066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.129:55649)
0619 17:02:02.751209 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:02.751262 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":67,"threads_started":1}
I20260619 17:02:02.855969 5831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:02.855704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.130:57181)
0619 17:02:02.855862 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:02.855908 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":64,"threads_started":1}
I20260619 17:02:03.002405 5832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:03.002070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.131:42441)
0619 17:02:03.002214 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:03.002258 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":71,"threads_started":1}
I20260619 17:02:03.223528 5833 raft_consensus.cc:482] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 17:02:03.223659 5833 raft_consensus.cc:504] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e45f8a53c1a54a84943d3641364ebc67" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46227 } } peers { permanent_uuid: "168e7085a6724eda937626d694c09a10" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40067 } } peers { permanent_uuid: "ea83fb233f30420b8639ba14498653c2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 41923 } }
I20260619 17:02:03.224081 5833 leader_election.cc:288] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e45f8a53c1a54a84943d3641364ebc67 (127.20.231.130:46227), 168e7085a6724eda937626d694c09a10 (127.20.231.131:40067)
Jun 19 17:02:03 dist-test-slave-vnbj krb5kdc[4250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888521, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for kudu/127.20.231.131@KRBTEST.COM
Jun 19 17:02:03 dist-test-slave-vnbj krb5kdc[4250](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888521, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for kudu/127.20.231.130@KRBTEST.COM
I20260619 17:02:03.237433 5834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:03.224318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:40067 (local address 127.20.231.129:45339)
0619 17:02:03.224466 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:03.224476 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:03.224522 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:03.225354 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:03.225357 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:03.225780 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:03.225983 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:03.225989 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:03.227187 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:03.227187 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:03.227994 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:03.227997 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:03.228664 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:03.228665 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:03.228738 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:03.229475 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:02:03.229484 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:03.231637 (+ 2153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:03.234676 (+ 3039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:03.234690 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:03.234693 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:03.235212 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:03.235636 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:03.235637 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:03.235638 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:03.235910 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:03.236509 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:03.236518 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:03.236876 (+ 358us) client_negotiation.cc:769] Sending connection context
0619 17:02:03.237047 (+ 171us) client_negotiation.cc:240] Negotiation successful
0619 17:02:03.237286 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":67,"thread_start_us":68,"threads_started":1}
I20260619 17:02:03.237463 5836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:03.224457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:59587 (local address 127.20.231.130:46227)
0619 17:02:03.224623 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:03.224631 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:03.224659 (+ 28us) server_negotiation.cc:407] Connection header received
0619 17:02:03.225098 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:03.225100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:03.225142 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:03.225297 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:03.226129 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:03.227065 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:03.228076 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:03.228514 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:03.228594 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:03.232050 (+ 3456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:03.232054 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:03.232055 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:03.232071 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:03.234512 (+ 2441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:03.235362 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:03.235366 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:03.235366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:03.235540 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:03.235989 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:03.235991 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:03.235992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:03.236374 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:03.236476 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:03.237163 (+ 687us) server_negotiation.cc:299] Negotiation successful
0619 17:02:03.237339 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":51,"threads_started":1}
I20260619 17:02:03.237480 5835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:03.224460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:45339 (local address 127.20.231.131:40067)
0619 17:02:03.224618 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:03.224630 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:03.224659 (+ 29us) server_negotiation.cc:407] Connection header received
0619 17:02:03.225098 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:03.225100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:03.225141 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:03.225317 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:03.226125 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:03.227073 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:03.228138 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:03.228537 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:03.228685 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:03.231808 (+ 3123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:03.231814 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:03.231815 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:03.231831 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:03.234507 (+ 2676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:03.235361 (+ 854us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:03.235366 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:03.235366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:03.235540 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:03.236023 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:03.236024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:03.236025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:03.236387 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:03.236476 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:03.237163 (+ 687us) server_negotiation.cc:299] Negotiation successful
0619 17:02:03.237377 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":45,"threads_started":1}
I20260619 17:02:03.237609 5830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:03.224291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:46227 (local address 127.20.231.129:59587)
0619 17:02:03.224386 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:03.224399 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:03.224459 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:03.225353 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:03.225356 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:03.225779 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:03.225983 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:03.225989 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:03.227168 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:03.227170 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:03.227976 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:03.227979 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:03.228615 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:03.228616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:03.228729 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:03.229475 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:02:03.229483 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:03.231881 (+ 2398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:03.234675 (+ 2794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:03.234690 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:03.234693 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:03.235192 (+ 499us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:03.235641 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:03.235642 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:03.235642 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:03.235834 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:03.236507 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:03.236518 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:03.236868 (+ 350us) client_negotiation.cc:769] Sending connection context
0619 17:02:03.237047 (+ 179us) client_negotiation.cc:240] Negotiation successful
0619 17:02:03.237421 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"mutex_wait_us":39}
W20260619 17:02:03.238600 5448 leader_election.cc:341] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 168e7085a6724eda937626d694c09a10 (127.20.231.131:40067): Not found: Tablet not found: a98453a99ca84f93be185abbb273a043
W20260619 17:02:03.239483 5447 leader_election.cc:341] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e45f8a53c1a54a84943d3641364ebc67 (127.20.231.130:46227): Not found: Tablet not found: a98453a99ca84f93be185abbb273a043
I20260619 17:02:03.239598 5447 leader_election.cc:302] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ea83fb233f30420b8639ba14498653c2; no voters: 168e7085a6724eda937626d694c09a10, e45f8a53c1a54a84943d3641364ebc67
I20260619 17:02:03.239797 5833 raft_consensus.cc:2726] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 17:02:03.752245 5837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:03.751990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.129:53797)
0619 17:02:03.752150 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:03.752197 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":110,"threads_started":1}
I20260619 17:02:03.856992 5838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:03.856556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.130:43187)
0619 17:02:03.856904 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:03.856945 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":90,"threads_started":1}
I20260619 17:02:04.003315 5839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:04.003028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.131:50517)
0619 17:02:04.003209 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:04.003259 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":77,"threads_started":1}
I20260619 17:02:04.753182 5840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:04.752895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.129:56963)
0619 17:02:04.753082 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:04.753132 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":70,"threads_started":1}
I20260619 17:02:04.857897 5841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:04.857667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.130:51117)
0619 17:02:04.857808 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:04.857851 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":63,"threads_started":1}
I20260619 17:02:05.004469 5842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:05.004142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.131:41631)
0619 17:02:05.004336 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:05.004418 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":71,"threads_started":1}
I20260619 17:02:05.450120 5843 raft_consensus.cc:482] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 17:02:05.450215 5843 raft_consensus.cc:504] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e45f8a53c1a54a84943d3641364ebc67" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46227 } } peers { permanent_uuid: "168e7085a6724eda937626d694c09a10" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40067 } } peers { permanent_uuid: "ea83fb233f30420b8639ba14498653c2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 41923 } }
I20260619 17:02:05.450505 5843 leader_election.cc:288] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e45f8a53c1a54a84943d3641364ebc67 (127.20.231.130:46227), 168e7085a6724eda937626d694c09a10 (127.20.231.131:40067)
W20260619 17:02:05.451115 5447 leader_election.cc:341] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e45f8a53c1a54a84943d3641364ebc67 (127.20.231.130:46227): Not found: Tablet not found: a98453a99ca84f93be185abbb273a043
W20260619 17:02:05.451223 5448 leader_election.cc:341] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 168e7085a6724eda937626d694c09a10 (127.20.231.131:40067): Not found: Tablet not found: a98453a99ca84f93be185abbb273a043
I20260619 17:02:05.451303 5447 leader_election.cc:302] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ea83fb233f30420b8639ba14498653c2; no voters: 168e7085a6724eda937626d694c09a10, e45f8a53c1a54a84943d3641364ebc67
I20260619 17:02:05.451397 5843 raft_consensus.cc:2726] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 17:02:05.754047 5844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:05.753755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.129:40361)
0619 17:02:05.753954 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:05.754000 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":78,"threads_started":1}
I20260619 17:02:05.858757 5845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:05.858509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.130:60023)
0619 17:02:05.858642 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:05.858690 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":62,"threads_started":1}
I20260619 17:02:06.005513 5846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:06.005224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.131:54143)
0619 17:02:06.005412 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:06.005462 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":80,"threads_started":1}
I20260619 17:02:06.755108 5847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:06.754825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.129:36781)
0619 17:02:06.755014 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:06.755059 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":122,"threads_started":1}
I20260619 17:02:06.859947 5848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:06.859522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.130:50323)
0619 17:02:06.859693 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:06.859854 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":89,"threads_started":1}
I20260619 17:02:07.006348 5849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:07.006080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.131:59703)
0619 17:02:07.006248 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:07.006296 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":65,"threads_started":1}
I20260619 17:02:07.756017 5850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:07.755752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.129:49603)
0619 17:02:07.755914 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:07.755959 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":110,"threads_started":1}
I20260619 17:02:07.860836 5851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:07.860521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.130:55447)
0619 17:02:07.860706 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:07.860785 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":85,"threads_started":1}
I20260619 17:02:08.007514 5852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:08.007135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.131:37925)
0619 17:02:08.007390 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:08.007445 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":65,"threads_started":1}
I20260619 17:02:08.675642 5853 raft_consensus.cc:482] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 17:02:08.675745 5853 raft_consensus.cc:504] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e45f8a53c1a54a84943d3641364ebc67" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46227 } } peers { permanent_uuid: "168e7085a6724eda937626d694c09a10" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40067 } } peers { permanent_uuid: "ea83fb233f30420b8639ba14498653c2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 41923 } }
I20260619 17:02:08.675932 5853 leader_election.cc:288] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e45f8a53c1a54a84943d3641364ebc67 (127.20.231.130:46227), 168e7085a6724eda937626d694c09a10 (127.20.231.131:40067)
W20260619 17:02:08.676463 5447 leader_election.cc:341] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e45f8a53c1a54a84943d3641364ebc67 (127.20.231.130:46227): Not found: Tablet not found: a98453a99ca84f93be185abbb273a043
W20260619 17:02:08.676587 5448 leader_election.cc:341] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 168e7085a6724eda937626d694c09a10 (127.20.231.131:40067): Not found: Tablet not found: a98453a99ca84f93be185abbb273a043
I20260619 17:02:08.676652 5448 leader_election.cc:302] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ea83fb233f30420b8639ba14498653c2; no voters: 168e7085a6724eda937626d694c09a10, e45f8a53c1a54a84943d3641364ebc67
I20260619 17:02:08.676733 5853 raft_consensus.cc:2726] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 17:02:08.757129 5854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:08.756672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.129:44087)
0619 17:02:08.757025 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:08.757079 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":79,"threads_started":1}
I20260619 17:02:08.861846 5855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:08.861492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.130:56561)
0619 17:02:08.861647 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:08.861706 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":82,"threads_started":1}
I20260619 17:02:09.008199 5856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:09.007965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.131:60409)
0619 17:02:09.008113 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:09.008151 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":58,"threads_started":1}
I20260619 17:02:09.758006 5857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:09.757792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.129:33907)
0619 17:02:09.757924 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:09.757962 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":70,"threads_started":1}
I20260619 17:02:09.862903 5858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:09.862514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.130:34355)
0619 17:02:09.862682 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:09.862771 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":82,"threads_started":1}
I20260619 17:02:10.009127 5859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:10.008809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.131:38417)
0619 17:02:10.009026 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:10.009081 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":87,"threads_started":1}
I20260619 17:02:10.490731 5860 raft_consensus.cc:482] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 17:02:10.490912 5860 raft_consensus.cc:504] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e45f8a53c1a54a84943d3641364ebc67" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46227 } } peers { permanent_uuid: "168e7085a6724eda937626d694c09a10" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40067 } } peers { permanent_uuid: "ea83fb233f30420b8639ba14498653c2" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 41923 } }
I20260619 17:02:10.491139 5860 leader_election.cc:288] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e45f8a53c1a54a84943d3641364ebc67 (127.20.231.130:46227), 168e7085a6724eda937626d694c09a10 (127.20.231.131:40067)
W20260619 17:02:10.491632 5447 leader_election.cc:341] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e45f8a53c1a54a84943d3641364ebc67 (127.20.231.130:46227): Not found: Tablet not found: a98453a99ca84f93be185abbb273a043
W20260619 17:02:10.491729 5448 leader_election.cc:341] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 168e7085a6724eda937626d694c09a10 (127.20.231.131:40067): Not found: Tablet not found: a98453a99ca84f93be185abbb273a043
I20260619 17:02:10.491906 5448 leader_election.cc:302] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ea83fb233f30420b8639ba14498653c2; no voters: 168e7085a6724eda937626d694c09a10, e45f8a53c1a54a84943d3641364ebc67
I20260619 17:02:10.492033 5860 raft_consensus.cc:2726] T a98453a99ca84f93be185abbb273a043 P ea83fb233f30420b8639ba14498653c2 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 17:02:10.758788 5861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:10.758510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.129:32989)
0619 17:02:10.758692 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:10.758740 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":69,"threads_started":1}
I20260619 17:02:10.863571 5862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:10.863316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.130:48407)
0619 17:02:10.863468 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:10.863507 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":80,"threads_started":1}
I20260619 17:02:11.010111 5863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:11.009813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.131:60383)
0619 17:02:11.010013 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:11.010062 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":72,"threads_started":1}
I20260619 17:02:11.759686 5864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:11.759433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.129:47861)
0619 17:02:11.759589 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:11.759639 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":70,"threads_started":1}
I20260619 17:02:11.864570 5865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:11.864244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.130:60011)
0619 17:02:11.864453 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:11.864506 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":132,"threads_started":1}
I20260619 17:02:12.011365 5866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:12.010988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.131:32893)
0619 17:02:12.011205 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:12.011315 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":127,"threads_started":1}
I20260619 17:02:12.760591 5867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:12.760317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.129:35789)
0619 17:02:12.760497 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:12.760545 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":93,"threads_started":1}
I20260619 17:02:12.865617 5868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:12.865335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.130:54575)
0619 17:02:12.865506 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:12.865559 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":81,"threads_started":1}
I20260619 17:02:13.012188 5869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:13.011931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.131:59249)
0619 17:02:13.012099 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:13.012135 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":78,"threads_started":1}
I20260619 17:02:13.761428 5870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:13.761174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.129:34435)
0619 17:02:13.761332 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:13.761382 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":67,"threads_started":1}
I20260619 17:02:13.866479 5871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:13.866242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.130:40853)
0619 17:02:13.866371 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:13.866431 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":62,"threads_started":1}
I20260619 17:02:14.012982 5872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:14.012727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.131:46389)
0619 17:02:14.012882 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:14.012932 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":68,"threads_started":1}
I20260619 17:02:14.762172 5873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:14.761956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.129:34301)
0619 17:02:14.762088 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:14.762125 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":62,"threads_started":1}
I20260619 17:02:14.867497 5874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:14.867260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.130:45351)
0619 17:02:14.867399 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:14.867449 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":60,"threads_started":1}
I20260619 17:02:15.013816 5875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:15.013577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.131:50011)
0619 17:02:15.013720 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:15.013769 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":76,"threads_started":1}
I20260619 17:02:15.762970 5876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:15.762671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.129:50651)
0619 17:02:15.762873 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:15.762918 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":128,"threads_started":1}
I20260619 17:02:15.868279 5877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:15.868037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.130:44277)
0619 17:02:15.868185 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:15.868229 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":69,"threads_started":1}
I20260619 17:02:16.014806 5878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:16.014415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.131:55599)
0619 17:02:16.014602 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:16.014759 (+ 157us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":130,"threads_started":1}
I20260619 17:02:16.763891 5879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:16.763656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.129:41135)
0619 17:02:16.763791 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:16.763840 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":60,"threads_started":1}
I20260619 17:02:16.869191 5880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:16.868907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.130:45855)
0619 17:02:16.869083 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:16.869134 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":70,"threads_started":1}
I20260619 17:02:17.015837 5881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:17.015546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:41493 (local address 127.20.231.131:58531)
0619 17:02:17.015721 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:17.015772 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.190:41493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":68,"threads_started":1}
I20260619 17:02:17.016536 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 5396
I20260619 17:02:17.021924 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 5563
I20260619 17:02:17.026770 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 5697
2026-06-19T17:02:17Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58682 ms)
[----------] 30 tests from SecurityITest (235691 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 17:02:17 dist-test-slave-vnbj krb5kdc[5884](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 17:02:17 dist-test-slave-vnbj krb5kdc[5884](info): set up 2 sockets
Jun 19 17:02:17 dist-test-slave-vnbj krb5kdc[5884](info): commencing operation
krb5kdc: starting...
W20260619 17:02:18.175168 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 17:02:18 dist-test-slave-vnbj krb5kdc[5884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T17:02:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T17:02:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.231.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
I20260619 17:02:18.312047 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:33271
--webserver_interface=127.20.231.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:33271
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:02:18.402626 5897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:18.402932 5897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:18.402978 5897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:18.404327 5897 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:02:18.404400 5897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:18.404417 5897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:18.404430 5897 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:02:18.404443 5897 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:02:18.406183 5897 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:33271
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:33271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:18.406424 5897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:18.406623 5897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:18.412266 5905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:18.412266 5902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:18.412305 5903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:18.412919 5897 server_base.cc:1034] running on GCE node
I20260619 17:02:18.413239 5897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:18.413616 5897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:18.414916 5897 hybrid_clock.cc:648] HybridClock initialized: now 1781888538414791 us; error 35 us; skew 500 ppm
Jun 19 17:02:18 dist-test-slave-vnbj krb5kdc[5884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888538, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:18.417661 5897 init.cc:377] Logged in from keytab as kudu/127.20.231.190@KRBTEST.COM (short username kudu)
I20260619 17:02:18.418232 5897 webserver.cc:460] Webserver started at http://127.20.231.190:35489/ using document root <none> and password file <none>
I20260619 17:02:18.418478 5897 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:18.418547 5897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:18.418707 5897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:18.419910 5897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "135171a3ecce407785e4de40d67285fd"
format_stamp: "Formatted at 2026-06-19 17:02:18 on dist-test-slave-vnbj"
server_key: "f1df4a237f556d550ea40c0ec2b34f35"
server_key_iv: "473c092c32d7d542091aabcf9f3fc6dd"
server_key_version: "encryptionkey@0"
I20260619 17:02:18.420320 5897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "135171a3ecce407785e4de40d67285fd"
format_stamp: "Formatted at 2026-06-19 17:02:18 on dist-test-slave-vnbj"
server_key: "f1df4a237f556d550ea40c0ec2b34f35"
server_key_iv: "473c092c32d7d542091aabcf9f3fc6dd"
server_key_version: "encryptionkey@0"
I20260619 17:02:18.421746 5897 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 17:02:18.422645 5912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:18.422993 5897 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:18.423102 5897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "135171a3ecce407785e4de40d67285fd"
format_stamp: "Formatted at 2026-06-19 17:02:18 on dist-test-slave-vnbj"
server_key: "f1df4a237f556d550ea40c0ec2b34f35"
server_key_iv: "473c092c32d7d542091aabcf9f3fc6dd"
server_key_version: "encryptionkey@0"
I20260619 17:02:18.423189 5897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:18.443590 5897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:18.446152 5897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:18.446332 5897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:18.450791 5897 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:33271
I20260619 17:02:18.450830 5964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:33271 every 8 connection(s)
I20260619 17:02:18.451263 5897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 17:02:18.452183 5965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:18.454643 5965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 135171a3ecce407785e4de40d67285fd: Bootstrap starting.
I20260619 17:02:18.455343 5965 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 135171a3ecce407785e4de40d67285fd: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:18.455659 5965 log.cc:824] T 00000000000000000000000000000000 P 135171a3ecce407785e4de40d67285fd: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:18.456377 5965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 135171a3ecce407785e4de40d67285fd: No bootstrap required, opened a new log
I20260619 17:02:18.458065 5965 raft_consensus.cc:348] T 00000000000000000000000000000000 P 135171a3ecce407785e4de40d67285fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135171a3ecce407785e4de40d67285fd" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 33271 } }
I20260619 17:02:18.458171 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 5897
I20260619 17:02:18.458237 5965 raft_consensus.cc:374] T 00000000000000000000000000000000 P 135171a3ecce407785e4de40d67285fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:18.458287 5965 raft_consensus.cc:729] T 00000000000000000000000000000000 P 135171a3ecce407785e4de40d67285fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 135171a3ecce407785e4de40d67285fd, State: Initialized, Role: FOLLOWER
I20260619 17:02:18.458310 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 17:02:18.458506 21406 external_mini_cluster.cc:1428] Setting key dbf56009557f477f248e2624e899651f
I20260619 17:02:18.458474 5965 consensus_queue.cc:260] T 00000000000000000000000000000000 P 135171a3ecce407785e4de40d67285fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135171a3ecce407785e4de40d67285fd" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 33271 } }
I20260619 17:02:18.458570 5965 raft_consensus.cc:388] T 00000000000000000000000000000000 P 135171a3ecce407785e4de40d67285fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 17:02:18.458612 5965 raft_consensus.cc:482] T 00000000000000000000000000000000 P 135171a3ecce407785e4de40d67285fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 17:02:18.458662 5965 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 135171a3ecce407785e4de40d67285fd [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:18.459537 5965 raft_consensus.cc:504] T 00000000000000000000000000000000 P 135171a3ecce407785e4de40d67285fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135171a3ecce407785e4de40d67285fd" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 33271 } }
I20260619 17:02:18.459658 5965 leader_election.cc:302] T 00000000000000000000000000000000 P 135171a3ecce407785e4de40d67285fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 135171a3ecce407785e4de40d67285fd; no voters:
I20260619 17:02:18.459817 5965 leader_election.cc:288] T 00000000000000000000000000000000 P 135171a3ecce407785e4de40d67285fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 17:02:18.459848 5970 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 135171a3ecce407785e4de40d67285fd [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:02:18.459964 5970 raft_consensus.cc:686] T 00000000000000000000000000000000 P 135171a3ecce407785e4de40d67285fd [term 1 LEADER]: Becoming Leader. State: Replica: 135171a3ecce407785e4de40d67285fd, State: Running, Role: LEADER
I20260619 17:02:18.460093 5965 sys_catalog.cc:564] T 00000000000000000000000000000000 P 135171a3ecce407785e4de40d67285fd [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:02:18.460081 5970 consensus_queue.cc:237] T 00000000000000000000000000000000 P 135171a3ecce407785e4de40d67285fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135171a3ecce407785e4de40d67285fd" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 33271 } }
I20260619 17:02:18.460455 5970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 135171a3ecce407785e4de40d67285fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "135171a3ecce407785e4de40d67285fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135171a3ecce407785e4de40d67285fd" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 33271 } } }
I20260619 17:02:18.460615 5970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 135171a3ecce407785e4de40d67285fd [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:18.460807 5970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 135171a3ecce407785e4de40d67285fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 135171a3ecce407785e4de40d67285fd. Latest consensus state: current_term: 1 leader_uuid: "135171a3ecce407785e4de40d67285fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "135171a3ecce407785e4de40d67285fd" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 33271 } } }
I20260619 17:02:18.460939 5970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 135171a3ecce407785e4de40d67285fd [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:18.461313 5978 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 17:02:18.461879 5978 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 17:02:18.463575 5978 catalog_manager.cc:1347] Generated new cluster ID: ec49a29209284e9fa077d9ac1c26822c
I20260619 17:02:18.463634 5978 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 17:02:18 dist-test-slave-vnbj krb5kdc[5884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888538, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:02:18.470521 5969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:18.459038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54960 (local address 127.20.231.190:33271)
0619 17:02:18.459716 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:18.459728 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:18.459765 (+ 37us) server_negotiation.cc:407] Connection header received
0619 17:02:18.459887 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:18.459913 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:18.459982 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:18.460955 (+ 973us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:18.461064 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:18.462464 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:18.463606 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:18.464107 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:18.464228 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:18.466767 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:18.466772 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:18.466772 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:18.466788 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:18.468737 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:18.469241 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:18.469245 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:18.469245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:18.469369 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:18.469601 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:18.469602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:18.469602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:18.469873 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:18.470058 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:18.470127 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 17:02:18.470294 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":535,"threads_started":1}
I20260619 17:02:18.499639 5978 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 17:02:18.500310 5978 catalog_manager.cc:1504] Loading token signing keys...
I20260619 17:02:18.507463 5978 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 135171a3ecce407785e4de40d67285fd: Generated new TSK 0
I20260619 17:02:18.507686 5978 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.231.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab.
Entry for principal kudu/127.20.231.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab.
I20260619 17:02:18.545336 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.129:0
--local_ip_for_outbound_sockets=127.20.231.129
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:33271
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:02:18.636603 5991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:18.636843 5991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:18.636870 5991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:18.638022 5991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:18.638062 5991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:18.638089 5991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.129
I20260619 17:02:18.639640 5991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:33271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.129
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:18.639871 5991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:18.640071 5991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:18.645921 5996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:18.645932 5997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:18.646051 5991 server_base.cc:1034] running on GCE node
W20260619 17:02:18.646003 5999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:18.646471 5991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:18.646757 5991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:18.647950 5991 hybrid_clock.cc:648] HybridClock initialized: now 1781888538647944 us; error 64 us; skew 500 ppm
Jun 19 17:02:18 dist-test-slave-vnbj krb5kdc[5884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888538, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:18.650924 5991 init.cc:377] Logged in from keytab as kudu/127.20.231.129@KRBTEST.COM (short username kudu)
I20260619 17:02:18.651499 5991 webserver.cc:460] Webserver started at http://127.20.231.129:36427/ using document root <none> and password file <none>
I20260619 17:02:18.651762 5991 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:18.651844 5991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:18.651997 5991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:18.653312 5991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-0/data/instance:
uuid: "ed76f6ce07f6481e88f1250a07aae2b9"
format_stamp: "Formatted at 2026-06-19 17:02:18 on dist-test-slave-vnbj"
server_key: "628e4eaa718d9b88e29f02875207285b"
server_key_iv: "18c9af94bfdb4fd2e328e39bd562bb2c"
server_key_version: "encryptionkey@0"
I20260619 17:02:18.653802 5991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance:
uuid: "ed76f6ce07f6481e88f1250a07aae2b9"
format_stamp: "Formatted at 2026-06-19 17:02:18 on dist-test-slave-vnbj"
server_key: "628e4eaa718d9b88e29f02875207285b"
server_key_iv: "18c9af94bfdb4fd2e328e39bd562bb2c"
server_key_version: "encryptionkey@0"
I20260619 17:02:18.655308 5991 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260619 17:02:18.656164 6006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:18.656400 5991 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:18.656512 5991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-0/wal
uuid: "ed76f6ce07f6481e88f1250a07aae2b9"
format_stamp: "Formatted at 2026-06-19 17:02:18 on dist-test-slave-vnbj"
server_key: "628e4eaa718d9b88e29f02875207285b"
server_key_iv: "18c9af94bfdb4fd2e328e39bd562bb2c"
server_key_version: "encryptionkey@0"
I20260619 17:02:18.656591 5991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:18.667987 5991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:18.670461 5991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:18.670662 5991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:18.671047 5991 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:02:18.671434 5991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:02:18.671492 5991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:18.671526 5991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:02:18.671558 5991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:18.677711 5991 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.129:32815
I20260619 17:02:18.677752 6119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.129:32815 every 8 connection(s)
I20260619 17:02:18.678117 5991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
I20260619 17:02:18.681669 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 5991
I20260619 17:02:18.681756 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance
I20260619 17:02:18.681980 21406 external_mini_cluster.cc:1428] Setting key 48a464805ba7b1a2c8b528ad782d0271
Jun 19 17:02:18 dist-test-slave-vnbj krb5kdc[5884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888538, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:02:18.689558 5969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:18.678983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:49355 (local address 127.20.231.190:33271)
0619 17:02:18.679056 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:18.679058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:18.679332 (+ 274us) server_negotiation.cc:407] Connection header received
0619 17:02:18.679958 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:18.679960 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:18.679974 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:18.680039 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:18.681181 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:18.681763 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:18.682980 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:18.683211 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:18.683291 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:18.686025 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:18.686026 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:18.686027 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:18.686034 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:18.687846 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:18.688404 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:18.688406 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:18.688406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:18.688480 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:18.688781 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:18.688782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:18.688782 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:18.688918 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:18.689004 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:18.689333 (+ 329us) server_negotiation.cc:299] Negotiation successful
0619 17:02:18.689437 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 17:02:18.689795 6122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:18.678962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33271 (local address 127.20.231.129:49355)
0619 17:02:18.679141 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:18.679185 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:18.679797 (+ 612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:18.680209 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:18.680214 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:18.680823 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:18.681047 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:18.681052 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:18.681879 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:18.681880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:18.682827 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:18.682830 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:18.683318 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:18.683318 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:18.683466 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:18.684126 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:02:18.684138 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:18.685852 (+ 1714us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:18.687988 (+ 2136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:18.687998 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:18.688001 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:18.688293 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:18.688565 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:18.688566 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:18.688567 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:18.688700 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:18.688994 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:18.689001 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:18.689171 (+ 170us) client_negotiation.cc:769] Sending connection context
0619 17:02:18.689308 (+ 137us) client_negotiation.cc:240] Negotiation successful
0619 17:02:18.689518 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":105,"threads_started":1}
I20260619 17:02:18.690637 6120 heartbeater.cc:347] Connected to a master server at 127.20.231.190:33271
I20260619 17:02:18.690759 6120 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.20.231.130@KRBTEST.COM; defaulting to no policy
I20260619 17:02:18.691007 6120 heartbeater.cc:511] Master 127.20.231.190:33271 requested a full tablet report, sending...
I20260619 17:02:18.691591 5929 ts_manager.cc:194] Registered new tserver with Master: ed76f6ce07f6481e88f1250a07aae2b9 (127.20.231.129:32815)
I20260619 17:02:18.692276 5929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.129@KRBTEST.COM'} at 127.20.231.129:49355
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab.
Entry for principal kudu/127.20.231.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab.
I20260619 17:02:18.710474 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.130:0
--local_ip_for_outbound_sockets=127.20.231.130
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:33271
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:02:18.796609 6125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:18.796909 6125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:18.796958 6125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:18.798264 6125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:18.798352 6125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:18.798399 6125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.130
I20260619 17:02:18.800029 6125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:33271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.130
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:18.800343 6125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:18.800613 6125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:18.807112 6133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:18.807112 6130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:18.807160 6131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:18.807466 6125 server_base.cc:1034] running on GCE node
I20260619 17:02:18.807679 6125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:18.807912 6125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:18.809060 6125 hybrid_clock.cc:648] HybridClock initialized: now 1781888538808991 us; error 87 us; skew 500 ppm
Jun 19 17:02:18 dist-test-slave-vnbj krb5kdc[5884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888538, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:18.811909 6125 init.cc:377] Logged in from keytab as kudu/127.20.231.130@KRBTEST.COM (short username kudu)
I20260619 17:02:18.812424 6125 webserver.cc:460] Webserver started at http://127.20.231.130:46789/ using document root <none> and password file <none>
I20260619 17:02:18.812655 6125 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:18.812723 6125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:18.812883 6125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:18.814015 6125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-1/data/instance:
uuid: "7498502ff17c460eb8728c4c59981b81"
format_stamp: "Formatted at 2026-06-19 17:02:18 on dist-test-slave-vnbj"
server_key: "552a70d342fc1696159f49f92607eeb8"
server_key_iv: "39a71ab0e36b65ca8a0bb54e84c8b682"
server_key_version: "encryptionkey@0"
I20260619 17:02:18.814394 6125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance:
uuid: "7498502ff17c460eb8728c4c59981b81"
format_stamp: "Formatted at 2026-06-19 17:02:18 on dist-test-slave-vnbj"
server_key: "552a70d342fc1696159f49f92607eeb8"
server_key_iv: "39a71ab0e36b65ca8a0bb54e84c8b682"
server_key_version: "encryptionkey@0"
I20260619 17:02:18.815951 6125 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 17:02:18.816735 6140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:18.816922 6125 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 17:02:18.817029 6125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-1/wal
uuid: "7498502ff17c460eb8728c4c59981b81"
format_stamp: "Formatted at 2026-06-19 17:02:18 on dist-test-slave-vnbj"
server_key: "552a70d342fc1696159f49f92607eeb8"
server_key_iv: "39a71ab0e36b65ca8a0bb54e84c8b682"
server_key_version: "encryptionkey@0"
I20260619 17:02:18.817104 6125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:18.837518 6125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:18.840253 6125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:18.840448 6125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:18.840721 6125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:02:18.841063 6125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:02:18.841120 6125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:18.841166 6125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:02:18.841192 6125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:18.847481 6125 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.130:46115
I20260619 17:02:18.847527 6253 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.130:46115 every 8 connection(s)
I20260619 17:02:18.847908 6125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
Jun 19 17:02:18 dist-test-slave-vnbj krb5kdc[5884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888538, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:02:18.856426 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 6125
I20260619 17:02:18.856561 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance
I20260619 17:02:18.856853 21406 external_mini_cluster.cc:1428] Setting key 7f005af968d63cbc3fb563d30c2dc492
I20260619 17:02:18.858492 5969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:18.848894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:55377 (local address 127.20.231.190:33271)
0619 17:02:18.849069 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:18.849071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:18.849160 (+ 89us) server_negotiation.cc:407] Connection header received
0619 17:02:18.849663 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:18.849664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:18.849677 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:18.849753 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:18.850809 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:18.851486 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:18.852582 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:18.852864 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:18.852933 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:18.855488 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:18.855489 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:18.855490 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:18.855497 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:18.856990 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:18.857462 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:18.857463 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:18.857464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:18.857525 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:18.857794 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:18.857795 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:18.857795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:18.857917 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:18.857989 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:18.858288 (+ 299us) server_negotiation.cc:299] Negotiation successful
0619 17:02:18.858389 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132}
I20260619 17:02:18.858687 6256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:18.848714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33271 (local address 127.20.231.130:55377)
0619 17:02:18.848989 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:18.849046 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:18.849559 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:18.849849 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:18.849853 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:18.850499 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:18.850709 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:18.850714 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:18.851589 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:18.851590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:18.852486 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:18.852488 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:18.852959 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:18.852960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:18.853074 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:18.853646 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:02:18.853658 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:18.855391 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:18.857092 (+ 1701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:18.857100 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:18.857103 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:18.857377 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:18.857593 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:18.857594 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:18.857594 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:18.857716 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:18.857990 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:18.857997 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:18.858149 (+ 152us) client_negotiation.cc:769] Sending connection context
0619 17:02:18.858257 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 17:02:18.858427 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":122,"threads_started":1}
I20260619 17:02:18.859170 6254 heartbeater.cc:347] Connected to a master server at 127.20.231.190:33271
I20260619 17:02:18.859299 6254 heartbeater.cc:464] Registering TS with master...
I20260619 17:02:18.859525 6254 heartbeater.cc:511] Master 127.20.231.190:33271 requested a full tablet report, sending...
I20260619 17:02:18.859930 5929 ts_manager.cc:194] Registered new tserver with Master: 7498502ff17c460eb8728c4c59981b81 (127.20.231.130:46115)
I20260619 17:02:18.860342 5929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.130@KRBTEST.COM'} at 127.20.231.130:55377
WARNING: no policy specified for kudu/127.20.231.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab.
Entry for principal kudu/127.20.231.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab.
I20260619 17:02:18.887504 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.131:0
--local_ip_for_outbound_sockets=127.20.231.131
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:33271
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:02:18.977102 6259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:18.977334 6259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:18.977360 6259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:18.978514 6259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:18.978556 6259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:18.978583 6259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.131
I20260619 17:02:18.980120 6259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:33271
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6259
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.131
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:18.980321 6259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:18.980511 6259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 17:02:18.986357 6259 server_base.cc:1034] running on GCE node
W20260619 17:02:18.986330 6264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:18.986356 6265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:18.986330 6267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:18.986932 6259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:18.987174 6259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:18.988323 6259 hybrid_clock.cc:648] HybridClock initialized: now 1781888538988312 us; error 35 us; skew 500 ppm
Jun 19 17:02:18 dist-test-slave-vnbj krb5kdc[5884](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888538, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:18.991443 6259 init.cc:377] Logged in from keytab as kudu/127.20.231.131@KRBTEST.COM (short username kudu)
I20260619 17:02:18.991988 6259 webserver.cc:460] Webserver started at http://127.20.231.131:33175/ using document root <none> and password file <none>
I20260619 17:02:18.992251 6259 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:18.992331 6259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:18.992516 6259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:18.993799 6259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-2/data/instance:
uuid: "cacd7332025d4f2b8b49d1e62625fe7f"
format_stamp: "Formatted at 2026-06-19 17:02:18 on dist-test-slave-vnbj"
server_key: "40acdace03cda7e1fb76012ab2929ca9"
server_key_iv: "18e0196b6db330215b92cec88c2ff0ef"
server_key_version: "encryptionkey@0"
I20260619 17:02:18.994225 6259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance:
uuid: "cacd7332025d4f2b8b49d1e62625fe7f"
format_stamp: "Formatted at 2026-06-19 17:02:18 on dist-test-slave-vnbj"
server_key: "40acdace03cda7e1fb76012ab2929ca9"
server_key_iv: "18e0196b6db330215b92cec88c2ff0ef"
server_key_version: "encryptionkey@0"
I20260619 17:02:18.995859 6259 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 17:02:18.996870 6274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:18.997097 6259 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:18.997222 6259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-2/wal
uuid: "cacd7332025d4f2b8b49d1e62625fe7f"
format_stamp: "Formatted at 2026-06-19 17:02:18 on dist-test-slave-vnbj"
server_key: "40acdace03cda7e1fb76012ab2929ca9"
server_key_iv: "18e0196b6db330215b92cec88c2ff0ef"
server_key_version: "encryptionkey@0"
I20260619 17:02:18.997308 6259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:19.016827 6259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:19.019254 6259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:19.019438 6259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:19.019706 6259 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:02:19.020051 6259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:02:19.020109 6259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:19.020150 6259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:02:19.020205 6259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:19.027586 6259 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.131:32977
I20260619 17:02:19.027629 6387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.131:32977 every 8 connection(s)
I20260619 17:02:19.028035 6259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
I20260619 17:02:19.033629 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 6259
I20260619 17:02:19.033718 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance
I20260619 17:02:19.033908 21406 external_mini_cluster.cc:1428] Setting key 6a86f0e429e78dcbd15c2b0098b8b683
Jun 19 17:02:19 dist-test-slave-vnbj krb5kdc[5884](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888538, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:02:19.039587 5969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:19.029040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:55411 (local address 127.20.231.190:33271)
0619 17:02:19.029120 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:19.029123 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:19.029331 (+ 208us) server_negotiation.cc:407] Connection header received
0619 17:02:19.029880 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:19.029881 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:19.029892 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:19.030023 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:19.031354 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.031925 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.033129 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.033326 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.033394 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:19.036315 (+ 2921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:19.036316 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:19.036317 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:19.036324 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:19.037856 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:19.038369 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:19.038370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:19.038370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:19.038435 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:19.038704 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:19.038705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:19.038705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:19.038823 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:19.039083 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:19.039435 (+ 352us) server_negotiation.cc:299] Negotiation successful
0619 17:02:19.039516 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 17:02:19.039909 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:19.028995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:33271 (local address 127.20.231.131:55411)
0619 17:02:19.029180 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:19.029229 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:19.029789 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:19.030107 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:19.030111 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:19.030764 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:19.031230 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:19.031237 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.032028 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.032028 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:19.033027 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:19.033030 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.033447 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.033449 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:19.033571 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:19.034366 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:02:19.034376 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:19.036204 (+ 1828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:19.038002 (+ 1798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:19.038010 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:19.038012 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:19.038254 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:19.038516 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:19.038517 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:19.038517 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:19.038640 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:19.039109 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:19.039117 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:19.039289 (+ 172us) client_negotiation.cc:769] Sending connection context
0619 17:02:19.039398 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 17:02:19.039546 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":105,"threads_started":1}
I20260619 17:02:19.040592 6388 heartbeater.cc:347] Connected to a master server at 127.20.231.190:33271
I20260619 17:02:19.040723 6388 heartbeater.cc:464] Registering TS with master...
I20260619 17:02:19.041040 6388 heartbeater.cc:511] Master 127.20.231.190:33271 requested a full tablet report, sending...
I20260619 17:02:19.041538 5929 ts_manager.cc:194] Registered new tserver with Master: cacd7332025d4f2b8b49d1e62625fe7f (127.20.231.131:32977)
I20260619 17:02:19.042307 5929 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.131@KRBTEST.COM'} at 127.20.231.131:55411
I20260619 17:02:19.046134 21406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 17:02:19.054322 5969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:19.047675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54964 (local address 127.20.231.190:33271)
0619 17:02:19.047733 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:19.047735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:19.047846 (+ 111us) server_negotiation.cc:407] Connection header received
0619 17:02:19.047898 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:19.047899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:19.047909 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:19.047963 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:19.048592 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.049130 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.050016 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.050208 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.050330 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:19.051611 (+ 1281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:19.051612 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:19.051613 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:19.051620 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:19.052984 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:19.053313 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:19.053314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:19.053315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:19.053368 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:19.053562 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:19.053563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:19.053563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:19.053713 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:19.053794 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:19.054127 (+ 333us) server_negotiation.cc:299] Negotiation successful
0619 17:02:19.054196 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 17:02:19.056118 5929 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54964:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 17:02:19.057744 5929 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 17:02:19.065775 6401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:19.060764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:46115 (local address 127.0.0.1:43242)
0619 17:02:19.061236 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:19.061247 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:19.061289 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:19.061948 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:19.061951 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:19.061956 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:02:19.062061 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:19.062065 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.063466 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.063467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:19.064603 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:19.064605 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.065326 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.065327 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:19.065415 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:19.065430 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 17:02:19.065566 (+ 136us) client_negotiation.cc:240] Negotiation successful
0619 17:02:19.065635 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"mutex_wait_us":76,"thread_start_us":97,"threads_started":1}
I20260619 17:02:19.065743 6403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:19.061090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43242 (local address 127.20.231.130:46115)
0619 17:02:19.061670 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:19.061675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:19.061685 (+ 10us) server_negotiation.cc:407] Connection header received
0619 17:02:19.061724 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:19.061726 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:19.061740 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:19.061844 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:02:19.062184 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.063368 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.064676 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.065228 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.065326 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:19.065388 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:19.065589 (+ 201us) server_negotiation.cc:299] Negotiation successful
0619 17:02:19.065635 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":109,"threads_started":1}
I20260619 17:02:19.066046 6402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:19.060912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39872 (local address 127.20.231.129:32815)
0619 17:02:19.062049 (+ 1137us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:19.062053 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:19.062063 (+ 10us) server_negotiation.cc:407] Connection header received
0619 17:02:19.062099 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:19.062101 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:19.062118 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:19.062194 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:02:19.062309 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.063088 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.064627 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.065339 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.065399 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:19.065446 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:19.065896 (+ 450us) server_negotiation.cc:299] Negotiation successful
0619 17:02:19.065960 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1100,"thread_start_us":87,"threads_started":1}
I20260619 17:02:19.066165 6404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:19.061360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36958 (local address 127.20.231.131:32977)
0619 17:02:19.061592 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:19.061596 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:19.061607 (+ 11us) server_negotiation.cc:407] Connection header received
0619 17:02:19.061699 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:19.061702 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:19.061720 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:19.061948 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:02:19.062149 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.063334 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.064538 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.065283 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.065362 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:19.065422 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:19.065699 (+ 277us) server_negotiation.cc:299] Negotiation successful
0619 17:02:19.065819 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":85,"threads_started":1}
I20260619 17:02:19.065841 6399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:19.060715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:32815 (local address 127.0.0.1:39872)
0619 17:02:19.061093 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:19.061170 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:19.061227 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:19.062190 (+ 963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:19.062191 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:19.062193 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:02:19.062245 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:19.062247 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.063194 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.063195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:19.064538 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:19.064541 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.065475 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.065475 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:19.065528 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:19.065533 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 17:02:19.065589 (+ 56us) client_negotiation.cc:240] Negotiation successful
0619 17:02:19.065654 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"mutex_wait_us":11,"thread_start_us":63,"threads_started":1}
I20260619 17:02:19.066049 6400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:19.060715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:32977 (local address 127.0.0.1:36958)
0619 17:02:19.060944 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:19.061040 (+ 96us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:19.061118 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:19.061948 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:19.061951 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:19.061954 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:02:19.062057 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:19.062060 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.063462 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.063463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:19.064432 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:19.064435 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.065460 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.065461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:19.065540 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:19.065544 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 17:02:19.065589 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 17:02:19.065652 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":9,"thread_start_us":91,"threads_started":1}
I20260619 17:02:19.067263 6322 tablet_service.cc:1467] Processing CreateTablet for tablet 8c1953aef8074fb38c431982874f73e1 (DEFAULT_TABLE table=test-table [id=3830dcf3e32340ffa289116621dd3102]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 17:02:19.067274 6054 tablet_service.cc:1467] Processing CreateTablet for tablet 8c1953aef8074fb38c431982874f73e1 (DEFAULT_TABLE table=test-table [id=3830dcf3e32340ffa289116621dd3102]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 17:02:19.067317 6188 tablet_service.cc:1467] Processing CreateTablet for tablet 8c1953aef8074fb38c431982874f73e1 (DEFAULT_TABLE table=test-table [id=3830dcf3e32340ffa289116621dd3102]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 17:02:19.067667 6322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c1953aef8074fb38c431982874f73e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:19.067673 6188 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c1953aef8074fb38c431982874f73e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:19.067674 6054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c1953aef8074fb38c431982874f73e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:19.070555 6405 tablet_bootstrap.cc:492] T 8c1953aef8074fb38c431982874f73e1 P cacd7332025d4f2b8b49d1e62625fe7f: Bootstrap starting.
I20260619 17:02:19.071223 6405 tablet_bootstrap.cc:654] T 8c1953aef8074fb38c431982874f73e1 P cacd7332025d4f2b8b49d1e62625fe7f: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:19.071417 6406 tablet_bootstrap.cc:492] T 8c1953aef8074fb38c431982874f73e1 P ed76f6ce07f6481e88f1250a07aae2b9: Bootstrap starting.
I20260619 17:02:19.072077 6405 log.cc:824] T 8c1953aef8074fb38c431982874f73e1 P cacd7332025d4f2b8b49d1e62625fe7f: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:19.072139 6406 tablet_bootstrap.cc:654] T 8c1953aef8074fb38c431982874f73e1 P ed76f6ce07f6481e88f1250a07aae2b9: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:19.072718 6406 log.cc:824] T 8c1953aef8074fb38c431982874f73e1 P ed76f6ce07f6481e88f1250a07aae2b9: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:19.073020 6405 tablet_bootstrap.cc:492] T 8c1953aef8074fb38c431982874f73e1 P cacd7332025d4f2b8b49d1e62625fe7f: No bootstrap required, opened a new log
I20260619 17:02:19.073098 6405 ts_tablet_manager.cc:1397] T 8c1953aef8074fb38c431982874f73e1 P cacd7332025d4f2b8b49d1e62625fe7f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 17:02:19.073098 6407 tablet_bootstrap.cc:492] T 8c1953aef8074fb38c431982874f73e1 P 7498502ff17c460eb8728c4c59981b81: Bootstrap starting.
I20260619 17:02:19.073463 6406 tablet_bootstrap.cc:492] T 8c1953aef8074fb38c431982874f73e1 P ed76f6ce07f6481e88f1250a07aae2b9: No bootstrap required, opened a new log
I20260619 17:02:19.073552 6406 ts_tablet_manager.cc:1397] T 8c1953aef8074fb38c431982874f73e1 P ed76f6ce07f6481e88f1250a07aae2b9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 17:02:19.073732 6407 tablet_bootstrap.cc:654] T 8c1953aef8074fb38c431982874f73e1 P 7498502ff17c460eb8728c4c59981b81: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:19.074316 6407 log.cc:824] T 8c1953aef8074fb38c431982874f73e1 P 7498502ff17c460eb8728c4c59981b81: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:19.075223 6407 tablet_bootstrap.cc:492] T 8c1953aef8074fb38c431982874f73e1 P 7498502ff17c460eb8728c4c59981b81: No bootstrap required, opened a new log
I20260619 17:02:19.075356 6407 ts_tablet_manager.cc:1397] T 8c1953aef8074fb38c431982874f73e1 P 7498502ff17c460eb8728c4c59981b81: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 17:02:19.075519 6405 raft_consensus.cc:348] T 8c1953aef8074fb38c431982874f73e1 P cacd7332025d4f2b8b49d1e62625fe7f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed76f6ce07f6481e88f1250a07aae2b9" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 32815 } } peers { permanent_uuid: "7498502ff17c460eb8728c4c59981b81" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46115 } } peers { permanent_uuid: "cacd7332025d4f2b8b49d1e62625fe7f" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 32977 } }
I20260619 17:02:19.075711 6405 raft_consensus.cc:374] T 8c1953aef8074fb38c431982874f73e1 P cacd7332025d4f2b8b49d1e62625fe7f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:19.075749 6405 raft_consensus.cc:729] T 8c1953aef8074fb38c431982874f73e1 P cacd7332025d4f2b8b49d1e62625fe7f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cacd7332025d4f2b8b49d1e62625fe7f, State: Initialized, Role: FOLLOWER
I20260619 17:02:19.075644 6406 raft_consensus.cc:348] T 8c1953aef8074fb38c431982874f73e1 P ed76f6ce07f6481e88f1250a07aae2b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed76f6ce07f6481e88f1250a07aae2b9" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 32815 } } peers { permanent_uuid: "7498502ff17c460eb8728c4c59981b81" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46115 } } peers { permanent_uuid: "cacd7332025d4f2b8b49d1e62625fe7f" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 32977 } }
I20260619 17:02:19.075860 6406 raft_consensus.cc:374] T 8c1953aef8074fb38c431982874f73e1 P ed76f6ce07f6481e88f1250a07aae2b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:19.075912 6406 raft_consensus.cc:729] T 8c1953aef8074fb38c431982874f73e1 P ed76f6ce07f6481e88f1250a07aae2b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed76f6ce07f6481e88f1250a07aae2b9, State: Initialized, Role: FOLLOWER
I20260619 17:02:19.075929 6405 consensus_queue.cc:260] T 8c1953aef8074fb38c431982874f73e1 P cacd7332025d4f2b8b49d1e62625fe7f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed76f6ce07f6481e88f1250a07aae2b9" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 32815 } } peers { permanent_uuid: "7498502ff17c460eb8728c4c59981b81" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46115 } } peers { permanent_uuid: "cacd7332025d4f2b8b49d1e62625fe7f" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 32977 } }
I20260619 17:02:19.076036 6406 consensus_queue.cc:260] T 8c1953aef8074fb38c431982874f73e1 P ed76f6ce07f6481e88f1250a07aae2b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed76f6ce07f6481e88f1250a07aae2b9" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 32815 } } peers { permanent_uuid: "7498502ff17c460eb8728c4c59981b81" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46115 } } peers { permanent_uuid: "cacd7332025d4f2b8b49d1e62625fe7f" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 32977 } }
I20260619 17:02:19.076344 6388 heartbeater.cc:503] Master 127.20.231.190:33271 was elected leader, sending a full tablet report...
I20260619 17:02:19.076346 6405 ts_tablet_manager.cc:1428] T 8c1953aef8074fb38c431982874f73e1 P cacd7332025d4f2b8b49d1e62625fe7f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 17:02:19.076468 6120 heartbeater.cc:503] Master 127.20.231.190:33271 was elected leader, sending a full tablet report...
I20260619 17:02:19.076710 6406 ts_tablet_manager.cc:1428] T 8c1953aef8074fb38c431982874f73e1 P ed76f6ce07f6481e88f1250a07aae2b9: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260619 17:02:19.077920 6407 raft_consensus.cc:348] T 8c1953aef8074fb38c431982874f73e1 P 7498502ff17c460eb8728c4c59981b81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed76f6ce07f6481e88f1250a07aae2b9" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 32815 } } peers { permanent_uuid: "7498502ff17c460eb8728c4c59981b81" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46115 } } peers { permanent_uuid: "cacd7332025d4f2b8b49d1e62625fe7f" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 32977 } }
I20260619 17:02:19.078063 6407 raft_consensus.cc:374] T 8c1953aef8074fb38c431982874f73e1 P 7498502ff17c460eb8728c4c59981b81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:19.078084 6407 raft_consensus.cc:729] T 8c1953aef8074fb38c431982874f73e1 P 7498502ff17c460eb8728c4c59981b81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7498502ff17c460eb8728c4c59981b81, State: Initialized, Role: FOLLOWER
I20260619 17:02:19.078166 6407 consensus_queue.cc:260] T 8c1953aef8074fb38c431982874f73e1 P 7498502ff17c460eb8728c4c59981b81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed76f6ce07f6481e88f1250a07aae2b9" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 32815 } } peers { permanent_uuid: "7498502ff17c460eb8728c4c59981b81" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46115 } } peers { permanent_uuid: "cacd7332025d4f2b8b49d1e62625fe7f" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 32977 } }
I20260619 17:02:19.078387 6407 ts_tablet_manager.cc:1428] T 8c1953aef8074fb38c431982874f73e1 P 7498502ff17c460eb8728c4c59981b81: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 17:02:19.078440 6254 heartbeater.cc:503] Master 127.20.231.190:33271 was elected leader, sending a full tablet report...
W20260619 17:02:19.098542 6255 tablet.cc:2369] T 8c1953aef8074fb38c431982874f73e1 P 7498502ff17c460eb8728c4c59981b81: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 17:02:19.179020 6121 tablet.cc:2369] T 8c1953aef8074fb38c431982874f73e1 P ed76f6ce07f6481e88f1250a07aae2b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 17:02:19.245744 6411 raft_consensus.cc:482] T 8c1953aef8074fb38c431982874f73e1 P cacd7332025d4f2b8b49d1e62625fe7f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 17:02:19.245942 6411 raft_consensus.cc:504] T 8c1953aef8074fb38c431982874f73e1 P cacd7332025d4f2b8b49d1e62625fe7f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed76f6ce07f6481e88f1250a07aae2b9" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 32815 } } peers { permanent_uuid: "7498502ff17c460eb8728c4c59981b81" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46115 } } peers { permanent_uuid: "cacd7332025d4f2b8b49d1e62625fe7f" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 32977 } }
I20260619 17:02:19.246351 6411 leader_election.cc:288] T 8c1953aef8074fb38c431982874f73e1 P cacd7332025d4f2b8b49d1e62625fe7f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed76f6ce07f6481e88f1250a07aae2b9 (127.20.231.129:32815), 7498502ff17c460eb8728c4c59981b81 (127.20.231.130:46115)
I20260619 17:02:19.249571 6403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:19.246794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:57053 (local address 127.20.231.130:46115)
0619 17:02:19.246834 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:19.246836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:19.246906 (+ 70us) server_negotiation.cc:407] Connection header received
0619 17:02:19.246938 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:19.246938 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:19.246949 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:19.247024 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:02:19.247208 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.247709 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.248729 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.249155 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.249251 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:19.249261 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:19.249439 (+ 178us) server_negotiation.cc:299] Negotiation successful
0619 17:02:19.249493 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 17:02:19.249572 6390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:19.246579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:46115 (local address 127.20.231.131:57053)
0619 17:02:19.246641 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:19.246654 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:19.246699 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:19.247040 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:19.247042 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:19.247043 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:02:19.247120 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:19.247121 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.247825 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.247826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:19.248623 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:19.248625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.249247 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.249248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:19.249338 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:19.249341 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 17:02:19.249425 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 17:02:19.249485 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 17:02:19.249788 6402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:19.246720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:45829 (local address 127.20.231.129:32815)
0619 17:02:19.246780 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:19.246783 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:19.247087 (+ 304us) server_negotiation.cc:407] Connection header received
0619 17:02:19.247160 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:19.247162 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:19.247183 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:19.247247 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:02:19.247431 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.247963 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.249012 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.249400 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.249500 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:19.249508 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:19.249681 (+ 173us) server_negotiation.cc:299] Negotiation successful
0619 17:02:19.249722 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 17:02:19.249799 6414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:19.246579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:32815 (local address 127.20.231.131:45829)
0619 17:02:19.246986 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:19.246998 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:19.247098 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:19.247261 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:19.247262 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:19.247263 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:02:19.247357 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:19.247359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.248059 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.248060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:19.248816 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:19.248818 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.249499 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.249500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:19.249574 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:19.249582 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 17:02:19.249672 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 17:02:19.249714 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":50,"thread_start_us":247,"threads_started":1}
I20260619 17:02:19.249990 6208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1953aef8074fb38c431982874f73e1" candidate_uuid: "cacd7332025d4f2b8b49d1e62625fe7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7498502ff17c460eb8728c4c59981b81" is_pre_election: true
I20260619 17:02:19.250182 6208 raft_consensus.cc:2454] T 8c1953aef8074fb38c431982874f73e1 P 7498502ff17c460eb8728c4c59981b81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cacd7332025d4f2b8b49d1e62625fe7f in term 0.
I20260619 17:02:19.250193 6074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1953aef8074fb38c431982874f73e1" candidate_uuid: "cacd7332025d4f2b8b49d1e62625fe7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed76f6ce07f6481e88f1250a07aae2b9" is_pre_election: true
I20260619 17:02:19.250373 6074 raft_consensus.cc:2454] T 8c1953aef8074fb38c431982874f73e1 P ed76f6ce07f6481e88f1250a07aae2b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cacd7332025d4f2b8b49d1e62625fe7f in term 0.
I20260619 17:02:19.250430 6277 leader_election.cc:302] T 8c1953aef8074fb38c431982874f73e1 P cacd7332025d4f2b8b49d1e62625fe7f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7498502ff17c460eb8728c4c59981b81, cacd7332025d4f2b8b49d1e62625fe7f; no voters:
I20260619 17:02:19.250579 6411 raft_consensus.cc:2781] T 8c1953aef8074fb38c431982874f73e1 P cacd7332025d4f2b8b49d1e62625fe7f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 17:02:19.250660 6411 raft_consensus.cc:482] T 8c1953aef8074fb38c431982874f73e1 P cacd7332025d4f2b8b49d1e62625fe7f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 17:02:19.250679 6411 raft_consensus.cc:3037] T 8c1953aef8074fb38c431982874f73e1 P cacd7332025d4f2b8b49d1e62625fe7f [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:19.251500 6411 raft_consensus.cc:504] T 8c1953aef8074fb38c431982874f73e1 P cacd7332025d4f2b8b49d1e62625fe7f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed76f6ce07f6481e88f1250a07aae2b9" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 32815 } } peers { permanent_uuid: "7498502ff17c460eb8728c4c59981b81" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46115 } } peers { permanent_uuid: "cacd7332025d4f2b8b49d1e62625fe7f" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 32977 } }
I20260619 17:02:19.251654 6411 leader_election.cc:288] T 8c1953aef8074fb38c431982874f73e1 P cacd7332025d4f2b8b49d1e62625fe7f [CANDIDATE]: Term 1 election: Requested vote from peers ed76f6ce07f6481e88f1250a07aae2b9 (127.20.231.129:32815), 7498502ff17c460eb8728c4c59981b81 (127.20.231.130:46115)
I20260619 17:02:19.251880 6208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1953aef8074fb38c431982874f73e1" candidate_uuid: "cacd7332025d4f2b8b49d1e62625fe7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7498502ff17c460eb8728c4c59981b81"
I20260619 17:02:19.251982 6208 raft_consensus.cc:3037] T 8c1953aef8074fb38c431982874f73e1 P 7498502ff17c460eb8728c4c59981b81 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:19.251993 6074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c1953aef8074fb38c431982874f73e1" candidate_uuid: "cacd7332025d4f2b8b49d1e62625fe7f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed76f6ce07f6481e88f1250a07aae2b9"
I20260619 17:02:19.252079 6074 raft_consensus.cc:3037] T 8c1953aef8074fb38c431982874f73e1 P ed76f6ce07f6481e88f1250a07aae2b9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:19.252836 6208 raft_consensus.cc:2454] T 8c1953aef8074fb38c431982874f73e1 P 7498502ff17c460eb8728c4c59981b81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cacd7332025d4f2b8b49d1e62625fe7f in term 1.
I20260619 17:02:19.252852 6074 raft_consensus.cc:2454] T 8c1953aef8074fb38c431982874f73e1 P ed76f6ce07f6481e88f1250a07aae2b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cacd7332025d4f2b8b49d1e62625fe7f in term 1.
I20260619 17:02:19.253044 6278 leader_election.cc:302] T 8c1953aef8074fb38c431982874f73e1 P cacd7332025d4f2b8b49d1e62625fe7f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cacd7332025d4f2b8b49d1e62625fe7f, ed76f6ce07f6481e88f1250a07aae2b9; no voters:
I20260619 17:02:19.253201 6411 raft_consensus.cc:2781] T 8c1953aef8074fb38c431982874f73e1 P cacd7332025d4f2b8b49d1e62625fe7f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:02:19.253420 6411 raft_consensus.cc:686] T 8c1953aef8074fb38c431982874f73e1 P cacd7332025d4f2b8b49d1e62625fe7f [term 1 LEADER]: Becoming Leader. State: Replica: cacd7332025d4f2b8b49d1e62625fe7f, State: Running, Role: LEADER
I20260619 17:02:19.253543 6411 consensus_queue.cc:237] T 8c1953aef8074fb38c431982874f73e1 P cacd7332025d4f2b8b49d1e62625fe7f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed76f6ce07f6481e88f1250a07aae2b9" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 32815 } } peers { permanent_uuid: "7498502ff17c460eb8728c4c59981b81" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46115 } } peers { permanent_uuid: "cacd7332025d4f2b8b49d1e62625fe7f" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 32977 } }
I20260619 17:02:19.254629 5928 catalog_manager.cc:5495] T 8c1953aef8074fb38c431982874f73e1 P cacd7332025d4f2b8b49d1e62625fe7f reported cstate change: term changed from 0 to 1, leader changed from <none> to cacd7332025d4f2b8b49d1e62625fe7f (127.20.231.131). New cstate: current_term: 1 leader_uuid: "cacd7332025d4f2b8b49d1e62625fe7f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed76f6ce07f6481e88f1250a07aae2b9" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 32815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7498502ff17c460eb8728c4c59981b81" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46115 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cacd7332025d4f2b8b49d1e62625fe7f" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 32977 } health_report { overall_health: HEALTHY } } }
W20260619 17:02:19.278659 6389 tablet.cc:2369] T 8c1953aef8074fb38c431982874f73e1 P cacd7332025d4f2b8b49d1e62625fe7f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 17:02:19.297812 6404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:19.295143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36974 (local address 127.20.231.131:32977)
0619 17:02:19.295212 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:19.295214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:19.295237 (+ 23us) server_negotiation.cc:407] Connection header received
0619 17:02:19.295286 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:19.295287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:19.295309 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:19.295399 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 17:02:19.295543 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.296116 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.296936 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.297142 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.297237 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:19.297424 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 17:02:19.297532 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 17:02:19.297634 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:19.297651 (+ 17us) server_negotiation.cc:299] Negotiation successful
0619 17:02:19.297674 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 17:02:19.299970 6074 raft_consensus.cc:1261] T 8c1953aef8074fb38c431982874f73e1 P ed76f6ce07f6481e88f1250a07aae2b9 [term 1 FOLLOWER]: Refusing update from remote peer cacd7332025d4f2b8b49d1e62625fe7f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 17:02:19.299970 6208 raft_consensus.cc:1261] T 8c1953aef8074fb38c431982874f73e1 P 7498502ff17c460eb8728c4c59981b81 [term 1 FOLLOWER]: Refusing update from remote peer cacd7332025d4f2b8b49d1e62625fe7f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 17:02:19.300372 6411 consensus_queue.cc:1035] T 8c1953aef8074fb38c431982874f73e1 P cacd7332025d4f2b8b49d1e62625fe7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed76f6ce07f6481e88f1250a07aae2b9" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 32815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 17:02:19.300580 6415 consensus_queue.cc:1035] T 8c1953aef8074fb38c431982874f73e1 P cacd7332025d4f2b8b49d1e62625fe7f [LEADER]: Connected to new peer: Peer: permanent_uuid: "7498502ff17c460eb8728c4c59981b81" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 46115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 17:02:19.303611 6419 mvcc.cc:205] Tried to move back new op lower bound from 7298615456969596928 to 7298615456783368192. Current Snapshot: MvccSnapshot[applied={T|T < 7298615456969596928}]
I20260619 17:02:19.303812 6418 mvcc.cc:205] Tried to move back new op lower bound from 7298615456969596928 to 7298615456783368192. Current Snapshot: MvccSnapshot[applied={T|T < 7298615456969596928 or (T in {7298615456969596928})}]
I20260619 17:02:19.305603 5928 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 17:02:19.305713 5928 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54964:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 17:02:19.306622 5928 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 135171a3ecce407785e4de40d67285fd: Sending DeleteTablet for 3 replicas of tablet 8c1953aef8074fb38c431982874f73e1
I20260619 17:02:19.307031 6188 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c1953aef8074fb38c431982874f73e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:02:19 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:43242
I20260619 17:02:19.307070 6054 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c1953aef8074fb38c431982874f73e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:02:19 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:39872
I20260619 17:02:19.307163 6322 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c1953aef8074fb38c431982874f73e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:02:19 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:36958
I20260619 17:02:19.307410 6428 tablet_replica.cc:330] T 8c1953aef8074fb38c431982874f73e1 P 7498502ff17c460eb8728c4c59981b81: stopping tablet replica
I20260619 17:02:19.307484 6429 tablet_replica.cc:330] T 8c1953aef8074fb38c431982874f73e1 P ed76f6ce07f6481e88f1250a07aae2b9: stopping tablet replica
I20260619 17:02:19.307523 6428 raft_consensus.cc:2229] T 8c1953aef8074fb38c431982874f73e1 P 7498502ff17c460eb8728c4c59981b81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 17:02:19.307564 6429 raft_consensus.cc:2229] T 8c1953aef8074fb38c431982874f73e1 P ed76f6ce07f6481e88f1250a07aae2b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 17:02:19.307626 6428 raft_consensus.cc:2258] T 8c1953aef8074fb38c431982874f73e1 P 7498502ff17c460eb8728c4c59981b81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 17:02:19.307631 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 5991
I20260619 17:02:19.307655 6429 raft_consensus.cc:2258] T 8c1953aef8074fb38c431982874f73e1 P ed76f6ce07f6481e88f1250a07aae2b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 17:02:19.308264 6430 tablet_replica.cc:330] T 8c1953aef8074fb38c431982874f73e1 P cacd7332025d4f2b8b49d1e62625fe7f: stopping tablet replica
I20260619 17:02:19.308593 6430 raft_consensus.cc:2229] T 8c1953aef8074fb38c431982874f73e1 P cacd7332025d4f2b8b49d1e62625fe7f [term 1 LEADER]: Raft consensus shutting down.
I20260619 17:02:19.308650 6428 ts_tablet_manager.cc:1905] T 8c1953aef8074fb38c431982874f73e1 P 7498502ff17c460eb8728c4c59981b81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 17:02:19.308734 6430 raft_consensus.cc:2258] T 8c1953aef8074fb38c431982874f73e1 P cacd7332025d4f2b8b49d1e62625fe7f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 17:02:19.310497 6428 ts_tablet_manager.cc:1918] T 8c1953aef8074fb38c431982874f73e1 P 7498502ff17c460eb8728c4c59981b81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 17:02:19.310578 6428 log.cc:1196] T 8c1953aef8074fb38c431982874f73e1 P 7498502ff17c460eb8728c4c59981b81: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-1/wal/wals/8c1953aef8074fb38c431982874f73e1
I20260619 17:02:19.310608 6430 ts_tablet_manager.cc:1905] T 8c1953aef8074fb38c431982874f73e1 P cacd7332025d4f2b8b49d1e62625fe7f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 17:02:19.311025 6428 ts_tablet_manager.cc:1939] T 8c1953aef8074fb38c431982874f73e1 P 7498502ff17c460eb8728c4c59981b81: Deleting consensus metadata
I20260619 17:02:19.312021 5915 catalog_manager.cc:4841] TS 7498502ff17c460eb8728c4c59981b81 (127.20.231.130:46115): tablet 8c1953aef8074fb38c431982874f73e1 (table test-table [id=3830dcf3e32340ffa289116621dd3102]) successfully deleted
I20260619 17:02:19.312433 6430 ts_tablet_manager.cc:1918] T 8c1953aef8074fb38c431982874f73e1 P cacd7332025d4f2b8b49d1e62625fe7f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 17:02:19.312533 6430 log.cc:1196] T 8c1953aef8074fb38c431982874f73e1 P cacd7332025d4f2b8b49d1e62625fe7f: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781888301345144-21406-0/minicluster-data/ts-2/wal/wals/8c1953aef8074fb38c431982874f73e1
I20260619 17:02:19.312783 6430 ts_tablet_manager.cc:1939] T 8c1953aef8074fb38c431982874f73e1 P cacd7332025d4f2b8b49d1e62625fe7f: Deleting consensus metadata
I20260619 17:02:19.313270 5913 catalog_manager.cc:4841] TS cacd7332025d4f2b8b49d1e62625fe7f (127.20.231.131:32977): tablet 8c1953aef8074fb38c431982874f73e1 (table test-table [id=3830dcf3e32340ffa289116621dd3102]) successfully deleted
W20260619 17:02:19.316697 5916 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.231.129:32815 (error 108)
I20260619 17:02:19.316901 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 6125
I20260619 17:02:19.317941 6399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:19.317705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:32815 (local address 127.0.0.1:39888)
0619 17:02:19.317767 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:19.317895 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.129:32815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260619 17:02:19.321856 5916 catalog_manager.cc:4584] TS ed76f6ce07f6481e88f1250a07aae2b9 (127.20.231.129:32815): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8c1953aef8074fb38c431982874f73e1: Network error: Client connection negotiation failed: client connection to 127.20.231.129:32815: connect: Connection refused (error 111)
I20260619 17:02:19.321983 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 6259
I20260619 17:02:19.326251 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 5897
2026-06-19T17:02:19Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2297 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-06-19T17:02:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T17:02:19Z Disabled control of system clock
I20260619 17:02:19.355175 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:39695
--webserver_interface=127.20.231.190
--webserver_port=0
--builtin_ntp_servers=127.20.231.148:43957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:39695
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 17:02:19.438709 6435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:19.439060 6435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:19.439086 6435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:19.440290 6435 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:02:19.440328 6435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:19.440341 6435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:19.440354 6435 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:02:19.440366 6435 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:02:19.441699 6435 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:43957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:39695
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:39695
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:19.441916 6435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:19.442162 6435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:19.447583 6440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:19.447575 6441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:19.447575 6443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:19.447764 6435 server_base.cc:1034] running on GCE node
I20260619 17:02:19.448135 6435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:19.448426 6435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:19.449584 6435 hybrid_clock.cc:648] HybridClock initialized: now 1781888539449551 us; error 44 us; skew 500 ppm
I20260619 17:02:19.451061 6435 webserver.cc:460] Webserver started at http://127.20.231.190:43637/ using document root <none> and password file <none>
I20260619 17:02:19.451328 6435 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:19.451453 6435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:19.451614 6435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:19.452731 6435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "33f89838dd964be586dda5a87be4415d"
format_stamp: "Formatted at 2026-06-19 17:02:19 on dist-test-slave-vnbj"
server_key: "b29a076bac9406897eaec65d641df059"
server_key_iv: "bc013832e5cf5fb7302eb99877e5889b"
server_key_version: "encryptionkey@0"
I20260619 17:02:19.453142 6435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "33f89838dd964be586dda5a87be4415d"
format_stamp: "Formatted at 2026-06-19 17:02:19 on dist-test-slave-vnbj"
server_key: "b29a076bac9406897eaec65d641df059"
server_key_iv: "bc013832e5cf5fb7302eb99877e5889b"
server_key_version: "encryptionkey@0"
I20260619 17:02:19.454644 6435 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 17:02:19.455813 6449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:19.455988 6435 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 17:02:19.456095 6435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "33f89838dd964be586dda5a87be4415d"
format_stamp: "Formatted at 2026-06-19 17:02:19 on dist-test-slave-vnbj"
server_key: "b29a076bac9406897eaec65d641df059"
server_key_iv: "bc013832e5cf5fb7302eb99877e5889b"
server_key_version: "encryptionkey@0"
I20260619 17:02:19.456177 6435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:19.493155 6435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:19.493544 6435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:19.493721 6435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:19.497951 6435 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:39695
I20260619 17:02:19.497989 6501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:39695 every 8 connection(s)
I20260619 17:02:19.498358 6435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 17:02:19.499594 6502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:19.500478 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 6435
I20260619 17:02:19.500571 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 17:02:19.500811 21406 external_mini_cluster.cc:1428] Setting key 98b02d4186be2ca35484ec774e37da73
I20260619 17:02:19.502318 6502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33f89838dd964be586dda5a87be4415d: Bootstrap starting.
I20260619 17:02:19.502944 6502 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33f89838dd964be586dda5a87be4415d: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:19.503245 6502 log.cc:824] T 00000000000000000000000000000000 P 33f89838dd964be586dda5a87be4415d: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:19.503955 6502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33f89838dd964be586dda5a87be4415d: No bootstrap required, opened a new log
I20260619 17:02:19.505798 6502 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33f89838dd964be586dda5a87be4415d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33f89838dd964be586dda5a87be4415d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 39695 } }
I20260619 17:02:19.505993 6502 raft_consensus.cc:374] T 00000000000000000000000000000000 P 33f89838dd964be586dda5a87be4415d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:19.506047 6502 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33f89838dd964be586dda5a87be4415d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33f89838dd964be586dda5a87be4415d, State: Initialized, Role: FOLLOWER
I20260619 17:02:19.506353 6502 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33f89838dd964be586dda5a87be4415d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33f89838dd964be586dda5a87be4415d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 39695 } }
I20260619 17:02:19.506599 6502 raft_consensus.cc:388] T 00000000000000000000000000000000 P 33f89838dd964be586dda5a87be4415d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 17:02:19.506641 6502 raft_consensus.cc:482] T 00000000000000000000000000000000 P 33f89838dd964be586dda5a87be4415d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 17:02:19.506695 6502 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33f89838dd964be586dda5a87be4415d [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:19.507200 6505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:19.501515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44174 (local address 127.20.231.190:39695)
0619 17:02:19.501739 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:19.501752 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:19.501780 (+ 28us) server_negotiation.cc:407] Connection header received
0619 17:02:19.502232 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:19.502240 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:19.502305 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:19.502493 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:19.502718 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.503865 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.504771 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.505236 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.505407 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:19.506288 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:19.506293 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:19.506294 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:19.506602 (+ 308us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:19.506655 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:19.506672 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:19.506742 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:19.506840 (+ 98us) server_negotiation.cc:299] Negotiation successful
0619 17:02:19.506952 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":108,"threads_started":1}
I20260619 17:02:19.507761 6502 raft_consensus.cc:504] T 00000000000000000000000000000000 P 33f89838dd964be586dda5a87be4415d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33f89838dd964be586dda5a87be4415d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 39695 } }
I20260619 17:02:19.507894 6502 leader_election.cc:302] T 00000000000000000000000000000000 P 33f89838dd964be586dda5a87be4415d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 33f89838dd964be586dda5a87be4415d; no voters:
I20260619 17:02:19.508105 6502 leader_election.cc:288] T 00000000000000000000000000000000 P 33f89838dd964be586dda5a87be4415d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 17:02:19.508294 6507 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 33f89838dd964be586dda5a87be4415d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:02:19.508369 6502 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33f89838dd964be586dda5a87be4415d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:02:19.508446 6507 raft_consensus.cc:686] T 00000000000000000000000000000000 P 33f89838dd964be586dda5a87be4415d [term 1 LEADER]: Becoming Leader. State: Replica: 33f89838dd964be586dda5a87be4415d, State: Running, Role: LEADER
I20260619 17:02:19.508545 6507 consensus_queue.cc:237] T 00000000000000000000000000000000 P 33f89838dd964be586dda5a87be4415d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33f89838dd964be586dda5a87be4415d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 39695 } }
I20260619 17:02:19.509042 6508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33f89838dd964be586dda5a87be4415d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "33f89838dd964be586dda5a87be4415d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33f89838dd964be586dda5a87be4415d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 39695 } } }
I20260619 17:02:19.509028 6509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33f89838dd964be586dda5a87be4415d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 33f89838dd964be586dda5a87be4415d. Latest consensus state: current_term: 1 leader_uuid: "33f89838dd964be586dda5a87be4415d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "33f89838dd964be586dda5a87be4415d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 39695 } } }
I20260619 17:02:19.509135 6508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33f89838dd964be586dda5a87be4415d [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:19.509142 6509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33f89838dd964be586dda5a87be4415d [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:19.509370 6516 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 17:02:19.509950 6516 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 17:02:19.511641 6516 catalog_manager.cc:1347] Generated new cluster ID: 4ac61b4dbb694b82be3374e647f0842f
I20260619 17:02:19.511698 6516 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 17:02:19.523476 6516 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 17:02:19.524181 6516 catalog_manager.cc:1504] Loading token signing keys...
I20260619 17:02:19.532032 6516 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 33f89838dd964be586dda5a87be4415d: Generated new TSK 0
I20260619 17:02:19.532265 6516 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 17:02:19.537732 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.129:0
--local_ip_for_outbound_sockets=127.20.231.129
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:39695
--builtin_ntp_servers=127.20.231.148:43957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 17:02:19.629077 6526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:19.629298 6526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:19.629323 6526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:19.630663 6526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:19.630724 6526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:19.630752 6526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.129
I20260619 17:02:19.633318 6526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:43957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:39695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.129
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:19.633627 6526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:19.634020 6526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 17:02:19.641470 6526 server_base.cc:1034] running on GCE node
W20260619 17:02:19.641486 6532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:19.641414 6534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:19.641417 6531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:19.641937 6526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:19.642223 6526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:19.643451 6526 hybrid_clock.cc:648] HybridClock initialized: now 1781888539643387 us; error 43 us; skew 500 ppm
I20260619 17:02:19.645082 6526 webserver.cc:460] Webserver started at http://127.20.231.129:46653/ using document root <none> and password file <none>
I20260619 17:02:19.645298 6526 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:19.645344 6526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:19.645442 6526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:19.646523 6526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-0/data/instance:
uuid: "72598100bc194f8b916c141ffe2530f0"
format_stamp: "Formatted at 2026-06-19 17:02:19 on dist-test-slave-vnbj"
server_key: "b93b64b503197823c7a41c0ffa2605b8"
server_key_iv: "b198bd3381383b663a238546dbede329"
server_key_version: "encryptionkey@0"
I20260619 17:02:19.646953 6526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance:
uuid: "72598100bc194f8b916c141ffe2530f0"
format_stamp: "Formatted at 2026-06-19 17:02:19 on dist-test-slave-vnbj"
server_key: "b93b64b503197823c7a41c0ffa2605b8"
server_key_iv: "b198bd3381383b663a238546dbede329"
server_key_version: "encryptionkey@0"
I20260619 17:02:19.648424 6526 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:19.649403 6540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:19.649794 6526 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:19.649993 6526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-0/wal
uuid: "72598100bc194f8b916c141ffe2530f0"
format_stamp: "Formatted at 2026-06-19 17:02:19 on dist-test-slave-vnbj"
server_key: "b93b64b503197823c7a41c0ffa2605b8"
server_key_iv: "b198bd3381383b663a238546dbede329"
server_key_version: "encryptionkey@0"
I20260619 17:02:19.650069 6526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:19.673946 6526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:19.674286 6526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:19.674451 6526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:19.674690 6526 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:02:19.675072 6526 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:02:19.675107 6526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:19.675130 6526 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:02:19.675174 6526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:19.681092 6526 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.129:33251
I20260619 17:02:19.681169 6653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.129:33251 every 8 connection(s)
I20260619 17:02:19.681550 6526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
I20260619 17:02:19.683944 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 6526
I20260619 17:02:19.684036 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance
I20260619 17:02:19.684253 21406 external_mini_cluster.cc:1428] Setting key 93114e9f29335209ed8e3625d00c2f92
I20260619 17:02:19.685297 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.130:0
--local_ip_for_outbound_sockets=127.20.231.130
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:39695
--builtin_ntp_servers=127.20.231.148:43957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 17:02:19.687791 6505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:19.682508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:39875 (local address 127.20.231.190:39695)
0619 17:02:19.682555 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:19.682557 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:19.682838 (+ 281us) server_negotiation.cc:407] Connection header received
0619 17:02:19.683463 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:19.683465 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:19.683482 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:19.683538 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:19.684388 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.685146 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.686236 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.686459 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.686533 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:19.687496 (+ 963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:19.687497 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:19.687498 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:19.687503 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:19.687519 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:19.687523 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:19.687577 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:19.687673 (+ 96us) server_negotiation.cc:299] Negotiation successful
0619 17:02:19.687711 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 17:02:19.688040 6656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:19.682489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:39695 (local address 127.20.231.129:39875)
0619 17:02:19.682711 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:19.682746 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:19.683334 (+ 588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:19.683632 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:19.683661 (+ 29us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:19.683670 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:19.684270 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:19.684275 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.685256 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.685257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:19.686142 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:19.686145 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.686555 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.686556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:19.686692 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:19.687326 (+ 634us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 17:02:19.687326 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:19.687364 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 17:02:19.687388 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:19.687580 (+ 192us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:19.687585 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:19.687589 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 17:02:19.687679 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 17:02:19.687785 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":99,"threads_started":1}
I20260619 17:02:19.688753 6654 heartbeater.cc:347] Connected to a master server at 127.20.231.190:39695
I20260619 17:02:19.688863 6654 heartbeater.cc:464] Registering TS with master...
I20260619 17:02:19.689071 6654 heartbeater.cc:511] Master 127.20.231.190:39695 requested a full tablet report, sending...
I20260619 17:02:19.689647 6466 ts_manager.cc:194] Registered new tserver with Master: 72598100bc194f8b916c141ffe2530f0 (127.20.231.129:33251)
I20260619 17:02:19.690270 6466 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.231.129:39875
W20260619 17:02:19.772202 6657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:19.772434 6657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:19.772459 6657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:19.773698 6657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:19.773746 6657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:19.773774 6657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.130
I20260619 17:02:19.775197 6657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:43957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:39695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.130
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:19.775504 6657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:19.775828 6657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:19.781247 6663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:19.781215 6665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:19.781388 6657 server_base.cc:1034] running on GCE node
W20260619 17:02:19.781237 6662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:19.781713 6657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:19.782020 6657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:19.783192 6657 hybrid_clock.cc:648] HybridClock initialized: now 1781888539783179 us; error 34 us; skew 500 ppm
I20260619 17:02:19.784513 6657 webserver.cc:460] Webserver started at http://127.20.231.130:42245/ using document root <none> and password file <none>
I20260619 17:02:19.784770 6657 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:19.784839 6657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:19.784994 6657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:19.786116 6657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-1/data/instance:
uuid: "39b4870b277744b99b996d722cdad065"
format_stamp: "Formatted at 2026-06-19 17:02:19 on dist-test-slave-vnbj"
server_key: "a8cff7105b4b177725d99075ec5f6c97"
server_key_iv: "29a3359446b6cd69ea74a55b3ceadf61"
server_key_version: "encryptionkey@0"
I20260619 17:02:19.786512 6657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance:
uuid: "39b4870b277744b99b996d722cdad065"
format_stamp: "Formatted at 2026-06-19 17:02:19 on dist-test-slave-vnbj"
server_key: "a8cff7105b4b177725d99075ec5f6c97"
server_key_iv: "29a3359446b6cd69ea74a55b3ceadf61"
server_key_version: "encryptionkey@0"
I20260619 17:02:19.788072 6657 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 17:02:19.788950 6671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:19.789192 6657 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 17:02:19.789314 6657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-1/wal
uuid: "39b4870b277744b99b996d722cdad065"
format_stamp: "Formatted at 2026-06-19 17:02:19 on dist-test-slave-vnbj"
server_key: "a8cff7105b4b177725d99075ec5f6c97"
server_key_iv: "29a3359446b6cd69ea74a55b3ceadf61"
server_key_version: "encryptionkey@0"
I20260619 17:02:19.789397 6657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:19.810626 6657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:19.811069 6657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:19.811236 6657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:19.811502 6657 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:02:19.811878 6657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:02:19.811946 6657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:19.811991 6657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:02:19.812018 6657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:19.818265 6657 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.130:45531
I20260619 17:02:19.818310 6784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.130:45531 every 8 connection(s)
I20260619 17:02:19.818706 6657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
I20260619 17:02:19.821156 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 6657
I20260619 17:02:19.821275 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance
I20260619 17:02:19.821470 21406 external_mini_cluster.cc:1428] Setting key 82e5dd3a71613d5d0ff3ba5fc67546bd
I20260619 17:02:19.822444 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.131:0
--local_ip_for_outbound_sockets=127.20.231.131
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:39695
--builtin_ntp_servers=127.20.231.148:43957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 17:02:19.824673 6505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:19.819592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:51067 (local address 127.20.231.190:39695)
0619 17:02:19.819658 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:19.819660 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:19.819931 (+ 271us) server_negotiation.cc:407] Connection header received
0619 17:02:19.820382 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:19.820383 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:19.820394 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:19.820463 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:19.821164 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.821884 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.823072 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.823315 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.823416 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:19.824366 (+ 950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:19.824368 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:19.824368 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:19.824374 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:19.824397 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:19.824402 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:19.824474 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:19.824561 (+ 87us) server_negotiation.cc:299] Negotiation successful
0619 17:02:19.824591 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 17:02:19.824937 6787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:19.819549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:39695 (local address 127.20.231.130:51067)
0619 17:02:19.819748 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:19.819790 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:19.820288 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:19.820522 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:19.820526 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:19.820534 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:19.821056 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:19.821060 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.821983 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.821984 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:19.822981 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:19.822984 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.823428 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.823429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:19.823586 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:19.824193 (+ 607us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 17:02:19.824194 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:19.824227 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 17:02:19.824254 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:19.824475 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:19.824482 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:19.824486 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 17:02:19.824564 (+ 78us) client_negotiation.cc:240] Negotiation successful
0619 17:02:19.824681 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":102,"threads_started":1}
I20260619 17:02:19.825439 6785 heartbeater.cc:347] Connected to a master server at 127.20.231.190:39695
I20260619 17:02:19.825538 6785 heartbeater.cc:464] Registering TS with master...
I20260619 17:02:19.825755 6785 heartbeater.cc:511] Master 127.20.231.190:39695 requested a full tablet report, sending...
I20260619 17:02:19.826211 6466 ts_manager.cc:194] Registered new tserver with Master: 39b4870b277744b99b996d722cdad065 (127.20.231.130:45531)
I20260619 17:02:19.826792 6466 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.231.130:51067
W20260619 17:02:19.906502 6788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:19.906737 6788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:19.906762 6788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:19.908265 6788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:19.908318 6788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:19.908347 6788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.131
I20260619 17:02:19.909991 6788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:43957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:39695
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6788
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.131
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:19.910269 6788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:19.910562 6788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:19.915928 6793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:19.915949 6794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:19.915992 6796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:19.916230 6788 server_base.cc:1034] running on GCE node
I20260619 17:02:19.916481 6788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:19.916759 6788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:19.917995 6788 hybrid_clock.cc:648] HybridClock initialized: now 1781888539917980 us; error 31 us; skew 500 ppm
I20260619 17:02:19.919597 6788 webserver.cc:460] Webserver started at http://127.20.231.131:44585/ using document root <none> and password file <none>
I20260619 17:02:19.919894 6788 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:19.919971 6788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:19.920122 6788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:19.921283 6788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-2/data/instance:
uuid: "8c24629ce18f44fea494a49df2b2c78e"
format_stamp: "Formatted at 2026-06-19 17:02:19 on dist-test-slave-vnbj"
server_key: "441791987f298ea1f404537ecdba58a1"
server_key_iv: "e197f015c18c407a85ba6a70eb2cc772"
server_key_version: "encryptionkey@0"
I20260619 17:02:19.921676 6788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance:
uuid: "8c24629ce18f44fea494a49df2b2c78e"
format_stamp: "Formatted at 2026-06-19 17:02:19 on dist-test-slave-vnbj"
server_key: "441791987f298ea1f404537ecdba58a1"
server_key_iv: "e197f015c18c407a85ba6a70eb2cc772"
server_key_version: "encryptionkey@0"
I20260619 17:02:19.923118 6788 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 17:02:19.924015 6802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:19.924206 6788 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:19.924316 6788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-2/wal
uuid: "8c24629ce18f44fea494a49df2b2c78e"
format_stamp: "Formatted at 2026-06-19 17:02:19 on dist-test-slave-vnbj"
server_key: "441791987f298ea1f404537ecdba58a1"
server_key_iv: "e197f015c18c407a85ba6a70eb2cc772"
server_key_version: "encryptionkey@0"
I20260619 17:02:19.924405 6788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:19.943707 6788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:19.944056 6788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:19.944236 6788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:19.944494 6788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:02:19.944844 6788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:02:19.944904 6788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:19.944949 6788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:02:19.944974 6788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:19.951310 6788 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.131:35053
I20260619 17:02:19.951598 6915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.131:35053 every 8 connection(s)
I20260619 17:02:19.951800 6788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
I20260619 17:02:19.957842 6505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:19.952694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:49671 (local address 127.20.231.190:39695)
0619 17:02:19.952864 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:19.952866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:19.953076 (+ 210us) server_negotiation.cc:407] Connection header received
0619 17:02:19.953651 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:19.953652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:19.953662 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:19.953729 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:19.954472 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.955040 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.956235 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.956460 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.956521 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:19.957609 (+ 1088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:19.957610 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:19.957611 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:19.957615 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:19.957628 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:19.957632 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:19.957678 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:19.957752 (+ 74us) server_negotiation.cc:299] Negotiation successful
0619 17:02:19.957775 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260619 17:02:19.958077 6918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:19.952666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:39695 (local address 127.20.231.131:49671)
0619 17:02:19.952932 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:19.952976 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:19.953545 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:19.953799 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:19.953803 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:19.953811 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:19.954351 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:19.954356 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.955162 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.955163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:19.956134 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:19.956137 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.956559 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.956560 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:19.956697 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:19.957458 (+ 761us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 17:02:19.957458 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:19.957492 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 17:02:19.957516 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:19.957678 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:19.957681 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:19.957684 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 17:02:19.957732 (+ 48us) client_negotiation.cc:240] Negotiation successful
0619 17:02:19.957836 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":153,"threads_started":1}
I20260619 17:02:19.958525 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 6788
I20260619 17:02:19.958628 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance
I20260619 17:02:19.958685 6916 heartbeater.cc:347] Connected to a master server at 127.20.231.190:39695
I20260619 17:02:19.958786 6916 heartbeater.cc:464] Registering TS with master...
I20260619 17:02:19.958870 21406 external_mini_cluster.cc:1428] Setting key 6e3dbbb25503a48bde2e7954e790728b
I20260619 17:02:19.959022 6916 heartbeater.cc:511] Master 127.20.231.190:39695 requested a full tablet report, sending...
I20260619 17:02:19.959383 6466 ts_manager.cc:194] Registered new tserver with Master: 8c24629ce18f44fea494a49df2b2c78e (127.20.231.131:35053)
I20260619 17:02:19.959897 6466 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.231.131:49671
I20260619 17:02:19.959888 21406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 17:02:19.964944 6505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:19.961571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44188 (local address 127.20.231.190:39695)
0619 17:02:19.961650 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:19.961652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:19.961750 (+ 98us) server_negotiation.cc:407] Connection header received
0619 17:02:19.961785 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:19.961786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:19.961794 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:19.961840 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:19.961974 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.962469 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.963379 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.963586 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.963645 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:19.964569 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:19.964570 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:19.964571 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:19.964575 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:19.964590 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:19.964594 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:19.964652 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:19.964823 (+ 171us) server_negotiation.cc:299] Negotiation successful
0619 17:02:19.964843 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 17:02:19.967010 6466 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44188:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 17:02:19.968653 6466 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 17:02:19.975888 6925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:19.971576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:45531 (local address 127.0.0.1:41754)
0619 17:02:19.971910 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:19.971919 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:19.971981 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:19.972243 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:19.972246 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:19.972248 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:02:19.972345 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:19.972348 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.973462 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.973464 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:19.974759 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:19.974761 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.975571 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.975572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:19.975657 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:19.975672 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 17:02:19.975747 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 17:02:19.975803 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":74,"threads_started":1}
I20260619 17:02:19.975899 6926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:19.971655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41754 (local address 127.20.231.130:45531)
0619 17:02:19.971956 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:19.971961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:19.971974 (+ 13us) server_negotiation.cc:407] Connection header received
0619 17:02:19.972105 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:19.972108 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:19.972126 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:19.972431 (+ 305us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:02:19.972493 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.973375 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.974900 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.975466 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.975560 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:19.975660 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:19.975763 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 17:02:19.975801 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":229,"threads_started":1}
I20260619 17:02:19.976058 6924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:19.971574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:35053 (local address 127.0.0.1:60460)
0619 17:02:19.971699 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:19.971743 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:19.971807 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:19.972355 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:19.972356 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:19.972358 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:02:19.972405 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:19.972406 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.973791 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.973792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:19.974828 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:19.974830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.975816 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.975817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:19.975877 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:19.975881 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 17:02:19.975913 (+ 32us) client_negotiation.cc:240] Negotiation successful
0619 17:02:19.975969 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":74,"threads_started":1}
I20260619 17:02:19.976177 6927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:19.971800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:33251 (local address 127.0.0.1:54898)
0619 17:02:19.972438 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:19.972448 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:19.972491 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:19.972769 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:19.972770 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:19.972772 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:02:19.972817 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:19.972818 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.974061 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.974062 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:19.974922 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:19.974925 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.975920 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.975921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:19.975969 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:19.975972 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 17:02:19.976019 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 17:02:19.976050 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":95,"threads_started":1}
I20260619 17:02:19.976773 6928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:19.971981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60460 (local address 127.20.231.131:35053)
0619 17:02:19.972166 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:19.972170 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:19.972182 (+ 12us) server_negotiation.cc:407] Connection header received
0619 17:02:19.972223 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:19.972225 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:19.972241 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:19.972355 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:02:19.972474 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.973694 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.974947 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.975711 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.976462 (+ 751us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:19.976549 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:19.976574 (+ 25us) server_negotiation.cc:299] Negotiation successful
0619 17:02:19.976691 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":101,"threads_started":1}
I20260619 17:02:19.976775 6719 tablet_service.cc:1467] Processing CreateTablet for tablet 8f50c2afb7284e1bb82d6e8d80d16cb8 (DEFAULT_TABLE table=test-table [id=2a308944c1074cb4b6db5b9b484f623d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 17:02:19.977306 6719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f50c2afb7284e1bb82d6e8d80d16cb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:19.980324 6930 tablet_bootstrap.cc:492] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 39b4870b277744b99b996d722cdad065: Bootstrap starting.
I20260619 17:02:19.976229 6929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:19.972260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54898 (local address 127.20.231.129:33251)
0619 17:02:19.972607 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:19.972611 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:19.972619 (+ 8us) server_negotiation.cc:407] Connection header received
0619 17:02:19.972656 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:19.972659 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:19.972673 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:19.972751 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:02:19.972912 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.973973 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.975006 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:19.975710 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:19.975800 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:19.975889 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:19.976100 (+ 211us) server_negotiation.cc:299] Negotiation successful
0619 17:02:19.976144 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":86,"threads_started":1}
I20260619 17:02:19.980760 6850 tablet_service.cc:1467] Processing CreateTablet for tablet 8f50c2afb7284e1bb82d6e8d80d16cb8 (DEFAULT_TABLE table=test-table [id=2a308944c1074cb4b6db5b9b484f623d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 17:02:19.981128 6850 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f50c2afb7284e1bb82d6e8d80d16cb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:19.981195 6930 tablet_bootstrap.cc:654] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 39b4870b277744b99b996d722cdad065: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:19.981288 6588 tablet_service.cc:1467] Processing CreateTablet for tablet 8f50c2afb7284e1bb82d6e8d80d16cb8 (DEFAULT_TABLE table=test-table [id=2a308944c1074cb4b6db5b9b484f623d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 17:02:19.981535 6930 log.cc:824] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 39b4870b277744b99b996d722cdad065: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:19.981630 6588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f50c2afb7284e1bb82d6e8d80d16cb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:19.982324 6930 tablet_bootstrap.cc:492] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 39b4870b277744b99b996d722cdad065: No bootstrap required, opened a new log
I20260619 17:02:19.982419 6930 ts_tablet_manager.cc:1397] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 39b4870b277744b99b996d722cdad065: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 17:02:19.984495 6930 raft_consensus.cc:348] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 39b4870b277744b99b996d722cdad065 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c24629ce18f44fea494a49df2b2c78e" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 35053 } } peers { permanent_uuid: "39b4870b277744b99b996d722cdad065" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45531 } } peers { permanent_uuid: "72598100bc194f8b916c141ffe2530f0" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 33251 } }
I20260619 17:02:19.984669 6930 raft_consensus.cc:374] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 39b4870b277744b99b996d722cdad065 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:19.984711 6930 raft_consensus.cc:729] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 39b4870b277744b99b996d722cdad065 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39b4870b277744b99b996d722cdad065, State: Initialized, Role: FOLLOWER
I20260619 17:02:19.984970 6932 tablet_bootstrap.cc:492] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 8c24629ce18f44fea494a49df2b2c78e: Bootstrap starting.
I20260619 17:02:19.985010 6930 consensus_queue.cc:260] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 39b4870b277744b99b996d722cdad065 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c24629ce18f44fea494a49df2b2c78e" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 35053 } } peers { permanent_uuid: "39b4870b277744b99b996d722cdad065" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45531 } } peers { permanent_uuid: "72598100bc194f8b916c141ffe2530f0" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 33251 } }
I20260619 17:02:19.985314 6930 ts_tablet_manager.cc:1428] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 39b4870b277744b99b996d722cdad065: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 17:02:19.985343 6785 heartbeater.cc:503] Master 127.20.231.190:39695 was elected leader, sending a full tablet report...
I20260619 17:02:19.985549 6933 tablet_bootstrap.cc:492] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 72598100bc194f8b916c141ffe2530f0: Bootstrap starting.
I20260619 17:02:19.985793 6932 tablet_bootstrap.cc:654] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 8c24629ce18f44fea494a49df2b2c78e: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:19.986151 6932 log.cc:824] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 8c24629ce18f44fea494a49df2b2c78e: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:19.986187 6933 tablet_bootstrap.cc:654] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 72598100bc194f8b916c141ffe2530f0: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:19.986502 6933 log.cc:824] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 72598100bc194f8b916c141ffe2530f0: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:19.987000 6932 tablet_bootstrap.cc:492] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 8c24629ce18f44fea494a49df2b2c78e: No bootstrap required, opened a new log
I20260619 17:02:19.987087 6932 ts_tablet_manager.cc:1397] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 8c24629ce18f44fea494a49df2b2c78e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 17:02:19.987418 6933 tablet_bootstrap.cc:492] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 72598100bc194f8b916c141ffe2530f0: No bootstrap required, opened a new log
I20260619 17:02:19.987481 6933 ts_tablet_manager.cc:1397] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 72598100bc194f8b916c141ffe2530f0: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 17:02:19.989147 6932 raft_consensus.cc:348] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 8c24629ce18f44fea494a49df2b2c78e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c24629ce18f44fea494a49df2b2c78e" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 35053 } } peers { permanent_uuid: "39b4870b277744b99b996d722cdad065" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45531 } } peers { permanent_uuid: "72598100bc194f8b916c141ffe2530f0" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 33251 } }
I20260619 17:02:19.989321 6932 raft_consensus.cc:374] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 8c24629ce18f44fea494a49df2b2c78e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:19.989368 6932 raft_consensus.cc:729] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 8c24629ce18f44fea494a49df2b2c78e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c24629ce18f44fea494a49df2b2c78e, State: Initialized, Role: FOLLOWER
I20260619 17:02:19.989404 6933 raft_consensus.cc:348] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 72598100bc194f8b916c141ffe2530f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c24629ce18f44fea494a49df2b2c78e" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 35053 } } peers { permanent_uuid: "39b4870b277744b99b996d722cdad065" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45531 } } peers { permanent_uuid: "72598100bc194f8b916c141ffe2530f0" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 33251 } }
I20260619 17:02:19.989509 6932 consensus_queue.cc:260] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 8c24629ce18f44fea494a49df2b2c78e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c24629ce18f44fea494a49df2b2c78e" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 35053 } } peers { permanent_uuid: "39b4870b277744b99b996d722cdad065" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45531 } } peers { permanent_uuid: "72598100bc194f8b916c141ffe2530f0" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 33251 } }
I20260619 17:02:19.989578 6933 raft_consensus.cc:374] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 72598100bc194f8b916c141ffe2530f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:19.989635 6933 raft_consensus.cc:729] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 72598100bc194f8b916c141ffe2530f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72598100bc194f8b916c141ffe2530f0, State: Initialized, Role: FOLLOWER
I20260619 17:02:19.989758 6933 consensus_queue.cc:260] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 72598100bc194f8b916c141ffe2530f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c24629ce18f44fea494a49df2b2c78e" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 35053 } } peers { permanent_uuid: "39b4870b277744b99b996d722cdad065" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45531 } } peers { permanent_uuid: "72598100bc194f8b916c141ffe2530f0" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 33251 } }
I20260619 17:02:19.989807 6932 ts_tablet_manager.cc:1428] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 8c24629ce18f44fea494a49df2b2c78e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 17:02:19.989823 6916 heartbeater.cc:503] Master 127.20.231.190:39695 was elected leader, sending a full tablet report...
I20260619 17:02:19.990025 6933 ts_tablet_manager.cc:1428] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 72598100bc194f8b916c141ffe2530f0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 17:02:19.990052 6654 heartbeater.cc:503] Master 127.20.231.190:39695 was elected leader, sending a full tablet report...
I20260619 17:02:20.012245 6938 raft_consensus.cc:482] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 72598100bc194f8b916c141ffe2530f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 17:02:20.012387 6938 raft_consensus.cc:504] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 72598100bc194f8b916c141ffe2530f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c24629ce18f44fea494a49df2b2c78e" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 35053 } } peers { permanent_uuid: "39b4870b277744b99b996d722cdad065" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45531 } } peers { permanent_uuid: "72598100bc194f8b916c141ffe2530f0" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 33251 } }
I20260619 17:02:20.012722 6938 leader_election.cc:288] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 72598100bc194f8b916c141ffe2530f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c24629ce18f44fea494a49df2b2c78e (127.20.231.131:35053), 39b4870b277744b99b996d722cdad065 (127.20.231.130:45531)
I20260619 17:02:20.016091 6656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:20.012951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:35053 (local address 127.20.231.129:58557)
0619 17:02:20.013015 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:20.013028 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:20.013085 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:20.013445 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:20.013447 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:20.013449 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:02:20.013530 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:20.013533 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:20.014234 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:20.014235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:20.015196 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:20.015199 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:20.015799 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:20.015800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:20.015878 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:20.015882 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 17:02:20.015937 (+ 55us) client_negotiation.cc:240] Negotiation successful
0619 17:02:20.015974 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260619 17:02:20.016095 6928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:20.013018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:58557 (local address 127.20.231.131:35053)
0619 17:02:20.013071 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:20.013074 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:20.013085 (+ 11us) server_negotiation.cc:407] Connection header received
0619 17:02:20.013201 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:20.013202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:20.013214 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:20.013297 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:02:20.013614 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:20.014144 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:20.015284 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:20.015713 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:20.015774 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:20.015781 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:20.015962 (+ 181us) server_negotiation.cc:299] Negotiation successful
0619 17:02:20.016017 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 17:02:20.016153 6926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:20.013086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:51769 (local address 127.20.231.130:45531)
0619 17:02:20.013145 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:20.013146 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:20.013217 (+ 71us) server_negotiation.cc:407] Connection header received
0619 17:02:20.013421 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:20.013422 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:20.013434 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:20.013520 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:02:20.013692 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:20.014270 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:20.015373 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:20.015742 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:20.015846 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:20.015853 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:20.016026 (+ 173us) server_negotiation.cc:299] Negotiation successful
0619 17:02:20.016075 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 17:02:20.016193 6939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:20.013002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:45531 (local address 127.20.231.129:51769)
0619 17:02:20.013149 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:20.013158 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:20.013248 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:20.013525 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:20.013525 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:20.013527 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:02:20.013615 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:20.013616 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:20.014373 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:20.014374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:20.015276 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:20.015279 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:20.015853 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:20.015854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:20.015937 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:20.015942 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 17:02:20.016001 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 17:02:20.016038 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":87,"threads_started":1}
I20260619 17:02:20.016552 6739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f50c2afb7284e1bb82d6e8d80d16cb8" candidate_uuid: "72598100bc194f8b916c141ffe2530f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39b4870b277744b99b996d722cdad065" is_pre_election: true
I20260619 17:02:20.016592 6870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f50c2afb7284e1bb82d6e8d80d16cb8" candidate_uuid: "72598100bc194f8b916c141ffe2530f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c24629ce18f44fea494a49df2b2c78e" is_pre_election: true
I20260619 17:02:20.016724 6739 raft_consensus.cc:2454] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 39b4870b277744b99b996d722cdad065 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72598100bc194f8b916c141ffe2530f0 in term 0.
I20260619 17:02:20.016749 6870 raft_consensus.cc:2454] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 8c24629ce18f44fea494a49df2b2c78e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 72598100bc194f8b916c141ffe2530f0 in term 0.
I20260619 17:02:20.017038 6541 leader_election.cc:302] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 72598100bc194f8b916c141ffe2530f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72598100bc194f8b916c141ffe2530f0, 8c24629ce18f44fea494a49df2b2c78e; no voters:
I20260619 17:02:20.017238 6938 raft_consensus.cc:2781] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 72598100bc194f8b916c141ffe2530f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 17:02:20.017292 6938 raft_consensus.cc:482] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 72598100bc194f8b916c141ffe2530f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 17:02:20.017357 6938 raft_consensus.cc:3037] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 72598100bc194f8b916c141ffe2530f0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:20.018201 6938 raft_consensus.cc:504] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 72598100bc194f8b916c141ffe2530f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c24629ce18f44fea494a49df2b2c78e" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 35053 } } peers { permanent_uuid: "39b4870b277744b99b996d722cdad065" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45531 } } peers { permanent_uuid: "72598100bc194f8b916c141ffe2530f0" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 33251 } }
I20260619 17:02:20.018358 6938 leader_election.cc:288] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 72598100bc194f8b916c141ffe2530f0 [CANDIDATE]: Term 1 election: Requested vote from peers 8c24629ce18f44fea494a49df2b2c78e (127.20.231.131:35053), 39b4870b277744b99b996d722cdad065 (127.20.231.130:45531)
I20260619 17:02:20.018507 6739 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f50c2afb7284e1bb82d6e8d80d16cb8" candidate_uuid: "72598100bc194f8b916c141ffe2530f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39b4870b277744b99b996d722cdad065"
I20260619 17:02:20.018582 6870 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f50c2afb7284e1bb82d6e8d80d16cb8" candidate_uuid: "72598100bc194f8b916c141ffe2530f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c24629ce18f44fea494a49df2b2c78e"
I20260619 17:02:20.018595 6739 raft_consensus.cc:3037] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 39b4870b277744b99b996d722cdad065 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:20.018663 6870 raft_consensus.cc:3037] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 8c24629ce18f44fea494a49df2b2c78e [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:20.019603 6739 raft_consensus.cc:2454] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 39b4870b277744b99b996d722cdad065 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72598100bc194f8b916c141ffe2530f0 in term 1.
I20260619 17:02:20.019603 6870 raft_consensus.cc:2454] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 8c24629ce18f44fea494a49df2b2c78e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 72598100bc194f8b916c141ffe2530f0 in term 1.
I20260619 17:02:20.019845 6541 leader_election.cc:302] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 72598100bc194f8b916c141ffe2530f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72598100bc194f8b916c141ffe2530f0, 8c24629ce18f44fea494a49df2b2c78e; no voters:
I20260619 17:02:20.019955 6938 raft_consensus.cc:2781] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 72598100bc194f8b916c141ffe2530f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:02:20.020109 6938 raft_consensus.cc:686] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 72598100bc194f8b916c141ffe2530f0 [term 1 LEADER]: Becoming Leader. State: Replica: 72598100bc194f8b916c141ffe2530f0, State: Running, Role: LEADER
I20260619 17:02:20.020224 6938 consensus_queue.cc:237] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 72598100bc194f8b916c141ffe2530f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c24629ce18f44fea494a49df2b2c78e" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 35053 } } peers { permanent_uuid: "39b4870b277744b99b996d722cdad065" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45531 } } peers { permanent_uuid: "72598100bc194f8b916c141ffe2530f0" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 33251 } }
I20260619 17:02:20.021140 6466 catalog_manager.cc:5495] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 72598100bc194f8b916c141ffe2530f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 72598100bc194f8b916c141ffe2530f0 (127.20.231.129). New cstate: current_term: 1 leader_uuid: "72598100bc194f8b916c141ffe2530f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c24629ce18f44fea494a49df2b2c78e" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 35053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "39b4870b277744b99b996d722cdad065" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45531 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "72598100bc194f8b916c141ffe2530f0" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 33251 } health_report { overall_health: HEALTHY } } }
I20260619 17:02:20.035211 6929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:20.032444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54904 (local address 127.20.231.129:33251)
0619 17:02:20.032511 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:20.032513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:20.032556 (+ 43us) server_negotiation.cc:407] Connection header received
0619 17:02:20.032602 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:20.032602 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:20.032628 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:20.032707 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 17:02:20.032866 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:20.033407 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:20.034236 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:20.034473 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:20.034539 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:20.034700 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 17:02:20.034823 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 17:02:20.035014 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:20.035061 (+ 47us) server_negotiation.cc:299] Negotiation successful
0619 17:02:20.035083 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 17:02:20.037331 6739 raft_consensus.cc:1261] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 39b4870b277744b99b996d722cdad065 [term 1 FOLLOWER]: Refusing update from remote peer 72598100bc194f8b916c141ffe2530f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 17:02:20.037456 6870 raft_consensus.cc:1261] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 8c24629ce18f44fea494a49df2b2c78e [term 1 FOLLOWER]: Refusing update from remote peer 72598100bc194f8b916c141ffe2530f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 17:02:20.037770 6940 consensus_queue.cc:1035] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 72598100bc194f8b916c141ffe2530f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39b4870b277744b99b996d722cdad065" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45531 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 17:02:20.037918 6938 consensus_queue.cc:1035] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 72598100bc194f8b916c141ffe2530f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c24629ce18f44fea494a49df2b2c78e" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 35053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 17:02:20.040443 6943 mvcc.cc:205] Tried to move back new op lower bound from 7298615459989590016 to 7298615459923431424. Current Snapshot: MvccSnapshot[applied={T|T < 7298615459989590016}]
I20260619 17:02:20.041348 6944 mvcc.cc:205] Tried to move back new op lower bound from 7298615459989590016 to 7298615459923431424. Current Snapshot: MvccSnapshot[applied={T|T < 7298615459989590016}]
I20260619 17:02:20.043190 6465 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 17:02:20.043322 6465 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 17:02:20.044178 6465 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 33f89838dd964be586dda5a87be4415d: Sending DeleteTablet for 3 replicas of tablet 8f50c2afb7284e1bb82d6e8d80d16cb8
I20260619 17:02:20.044579 6719 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f50c2afb7284e1bb82d6e8d80d16cb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:02:20 UTC) from {username='slave'} at 127.0.0.1:41754
I20260619 17:02:20.044651 6850 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f50c2afb7284e1bb82d6e8d80d16cb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:02:20 UTC) from {username='slave'} at 127.0.0.1:60460
I20260619 17:02:20.044836 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 6526
I20260619 17:02:20.044866 6588 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f50c2afb7284e1bb82d6e8d80d16cb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:02:20 UTC) from {username='slave'} at 127.0.0.1:54898
I20260619 17:02:20.046917 6954 tablet_replica.cc:330] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 8c24629ce18f44fea494a49df2b2c78e: stopping tablet replica
I20260619 17:02:20.047036 6954 raft_consensus.cc:2229] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 8c24629ce18f44fea494a49df2b2c78e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 17:02:20.047129 6954 raft_consensus.cc:2258] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 8c24629ce18f44fea494a49df2b2c78e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 17:02:20.047562 6954 ts_tablet_manager.cc:1905] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 8c24629ce18f44fea494a49df2b2c78e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 17:02:20.049223 6954 ts_tablet_manager.cc:1918] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 8c24629ce18f44fea494a49df2b2c78e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 17:02:20.049295 6954 log.cc:1196] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 8c24629ce18f44fea494a49df2b2c78e: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781888301345144-21406-0/minicluster-data/ts-2/wal/wals/8f50c2afb7284e1bb82d6e8d80d16cb8
I20260619 17:02:20.049612 6954 ts_tablet_manager.cc:1939] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 8c24629ce18f44fea494a49df2b2c78e: Deleting consensus metadata
W20260619 17:02:20.050086 6450 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.231.129:33251 (error 108)
I20260619 17:02:20.050266 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 6657
I20260619 17:02:20.050303 6953 tablet_replica.cc:330] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 39b4870b277744b99b996d722cdad065: stopping tablet replica
I20260619 17:02:20.050444 6953 raft_consensus.cc:2229] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 39b4870b277744b99b996d722cdad065 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 17:02:20.050554 6953 raft_consensus.cc:2258] T 8f50c2afb7284e1bb82d6e8d80d16cb8 P 39b4870b277744b99b996d722cdad065 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 17:02:20.051363 6450 catalog_manager.cc:4841] TS 8c24629ce18f44fea494a49df2b2c78e (127.20.231.131:35053): tablet 8f50c2afb7284e1bb82d6e8d80d16cb8 (table test-table [id=2a308944c1074cb4b6db5b9b484f623d]) successfully deleted
I20260619 17:02:20.055516 6927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:20.055309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:33251 (local address 127.0.0.1:54912)
0619 17:02:20.055378 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:20.055463 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.129:33251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260619 17:02:20.055518 6925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:20.055417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:45531 (local address 127.0.0.1:41762)
0619 17:02:20.055476 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:20.055502 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.130:45531: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260619 17:02:20.055629 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 6788
W20260619 17:02:20.055836 6450 catalog_manager.cc:4584] TS 72598100bc194f8b916c141ffe2530f0 (127.20.231.129:33251): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8f50c2afb7284e1bb82d6e8d80d16cb8: Network error: Client connection negotiation failed: client connection to 127.20.231.129:33251: connect: Connection refused (error 111)
I20260619 17:02:20.060619 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 6435
2026-06-19T17:02:20Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (734 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 17:02:20 dist-test-slave-vnbj krb5kdc[6958](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 17:02:20 dist-test-slave-vnbj krb5kdc[6958](info): set up 2 sockets
Jun 19 17:02:20 dist-test-slave-vnbj krb5kdc[6958](info): commencing operation
krb5kdc: starting...
W20260619 17:02:21.166712 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 17:02:21 dist-test-slave-vnbj krb5kdc[6958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T17:02:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T17:02:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.231.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
I20260619 17:02:21.299927 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:34085
--webserver_interface=127.20.231.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:44643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:34085
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:02:21.386449 6971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:21.386693 6971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:21.386726 6971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:21.387972 6971 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:02:21.388026 6971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:21.388043 6971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:21.388058 6971 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:02:21.388073 6971 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:02:21.389668 6971 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:44643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:34085
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:34085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:21.389953 6971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:21.390223 6971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:21.395612 6977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:21.395641 6979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:21.395651 6976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:21.395725 6971 server_base.cc:1034] running on GCE node
I20260619 17:02:21.396196 6971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:21.396600 6971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:21.397764 6971 hybrid_clock.cc:648] HybridClock initialized: now 1781888541397712 us; error 66 us; skew 500 ppm
Jun 19 17:02:21 dist-test-slave-vnbj krb5kdc[6958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888541, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:21.400667 6971 init.cc:377] Logged in from keytab as kudu/127.20.231.190@KRBTEST.COM (short username kudu)
I20260619 17:02:21.401226 6971 webserver.cc:460] Webserver started at http://127.20.231.190:36669/ using document root <none> and password file <none>
I20260619 17:02:21.401481 6971 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:21.401556 6971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:21.401708 6971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:21.402941 6971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "69e533a86cf3419e9fc6199256b2bcae"
format_stamp: "Formatted at 2026-06-19 17:02:21 on dist-test-slave-vnbj"
server_key: "37bb3b0ecde2cb5ef79e0df0535af9e6"
server_key_iv: "82941bee497396b17108f29032ddf136"
server_key_version: "encryptionkey@0"
I20260619 17:02:21.403318 6971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "69e533a86cf3419e9fc6199256b2bcae"
format_stamp: "Formatted at 2026-06-19 17:02:21 on dist-test-slave-vnbj"
server_key: "37bb3b0ecde2cb5ef79e0df0535af9e6"
server_key_iv: "82941bee497396b17108f29032ddf136"
server_key_version: "encryptionkey@0"
I20260619 17:02:21.404735 6971 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 17:02:21.405660 6986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:21.405905 6971 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 17:02:21.406067 6971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "69e533a86cf3419e9fc6199256b2bcae"
format_stamp: "Formatted at 2026-06-19 17:02:21 on dist-test-slave-vnbj"
server_key: "37bb3b0ecde2cb5ef79e0df0535af9e6"
server_key_iv: "82941bee497396b17108f29032ddf136"
server_key_version: "encryptionkey@0"
I20260619 17:02:21.406150 6971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:21.429641 6971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:21.432113 6971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:21.432301 6971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:21.436654 6971 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:34085
I20260619 17:02:21.436692 7038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:34085 every 8 connection(s)
I20260619 17:02:21.437065 6971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 17:02:21.438002 7039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:21.440358 7039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69e533a86cf3419e9fc6199256b2bcae: Bootstrap starting.
I20260619 17:02:21.440960 7039 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69e533a86cf3419e9fc6199256b2bcae: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:21.441258 7039 log.cc:824] T 00000000000000000000000000000000 P 69e533a86cf3419e9fc6199256b2bcae: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:21.442054 7039 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69e533a86cf3419e9fc6199256b2bcae: No bootstrap required, opened a new log
I20260619 17:02:21.443886 7039 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69e533a86cf3419e9fc6199256b2bcae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69e533a86cf3419e9fc6199256b2bcae" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 34085 } }
I20260619 17:02:21.444031 7039 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69e533a86cf3419e9fc6199256b2bcae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:21.444054 7039 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69e533a86cf3419e9fc6199256b2bcae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69e533a86cf3419e9fc6199256b2bcae, State: Initialized, Role: FOLLOWER
I20260619 17:02:21.444204 7039 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69e533a86cf3419e9fc6199256b2bcae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69e533a86cf3419e9fc6199256b2bcae" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 34085 } }
I20260619 17:02:21.444293 7039 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69e533a86cf3419e9fc6199256b2bcae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 17:02:21.444317 7039 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69e533a86cf3419e9fc6199256b2bcae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 17:02:21.444386 7039 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69e533a86cf3419e9fc6199256b2bcae [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:21.444959 7039 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69e533a86cf3419e9fc6199256b2bcae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69e533a86cf3419e9fc6199256b2bcae" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 34085 } }
I20260619 17:02:21.445046 7039 leader_election.cc:302] T 00000000000000000000000000000000 P 69e533a86cf3419e9fc6199256b2bcae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69e533a86cf3419e9fc6199256b2bcae; no voters:
I20260619 17:02:21.445240 7039 leader_election.cc:288] T 00000000000000000000000000000000 P 69e533a86cf3419e9fc6199256b2bcae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 17:02:21.445375 7042 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69e533a86cf3419e9fc6199256b2bcae [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:02:21.445529 7039 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69e533a86cf3419e9fc6199256b2bcae [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:02:21.445569 7042 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69e533a86cf3419e9fc6199256b2bcae [term 1 LEADER]: Becoming Leader. State: Replica: 69e533a86cf3419e9fc6199256b2bcae, State: Running, Role: LEADER
I20260619 17:02:21.445686 7042 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69e533a86cf3419e9fc6199256b2bcae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69e533a86cf3419e9fc6199256b2bcae" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 34085 } }
I20260619 17:02:21.445844 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 6971
I20260619 17:02:21.445951 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 17:02:21.446171 21406 external_mini_cluster.cc:1428] Setting key 1d911124e7c8e174ddb427da7970d3cc
I20260619 17:02:21.446285 7044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69e533a86cf3419e9fc6199256b2bcae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69e533a86cf3419e9fc6199256b2bcae. Latest consensus state: current_term: 1 leader_uuid: "69e533a86cf3419e9fc6199256b2bcae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69e533a86cf3419e9fc6199256b2bcae" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 34085 } } }
I20260619 17:02:21.446422 7044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69e533a86cf3419e9fc6199256b2bcae [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:21.446431 7043 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69e533a86cf3419e9fc6199256b2bcae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69e533a86cf3419e9fc6199256b2bcae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69e533a86cf3419e9fc6199256b2bcae" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 34085 } } }
I20260619 17:02:21.446539 7043 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69e533a86cf3419e9fc6199256b2bcae [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:21.447665 7051 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 17:02:21.448264 7051 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 17:02:21.450081 7051 catalog_manager.cc:1347] Generated new cluster ID: b2f215ac6027429bb52e05cce379d98b
I20260619 17:02:21.450151 7051 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 17:02:21 dist-test-slave-vnbj krb5kdc[6958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888541, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:02:21.459293 7049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:21.447081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55754 (local address 127.20.231.190:34085)
0619 17:02:21.447414 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:21.447424 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:21.447462 (+ 38us) server_negotiation.cc:407] Connection header received
0619 17:02:21.447596 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:21.447604 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:21.447665 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:21.447930 (+ 265us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:21.448740 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:21.450034 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:21.451150 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:21.451822 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:21.451934 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:21.455047 (+ 3113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:21.455053 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:21.455053 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:21.455068 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:21.457326 (+ 2258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:21.457794 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:21.457797 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:21.457798 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:21.457939 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:21.458176 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:21.458177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:21.458177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:21.458532 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:21.458690 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:21.458888 (+ 198us) server_negotiation.cc:299] Negotiation successful
0619 17:02:21.459035 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":99,"threads_started":1}
I20260619 17:02:21.469956 7051 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 17:02:21.470713 7051 catalog_manager.cc:1504] Loading token signing keys...
I20260619 17:02:21.479460 7051 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69e533a86cf3419e9fc6199256b2bcae: Generated new TSK 0
I20260619 17:02:21.479753 7051 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.231.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab.
Entry for principal kudu/127.20.231.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab.
I20260619 17:02:21.514062 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.129:0
--local_ip_for_outbound_sockets=127.20.231.129
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:34085
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:44643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:02:21.597544 7065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:21.597784 7065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:21.597832 7065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:21.599113 7065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:21.599176 7065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:21.599220 7065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.129
I20260619 17:02:21.600740 7065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:44643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:34085
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.129
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:21.601018 7065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:21.601262 7065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 17:02:21.607025 7065 server_base.cc:1034] running on GCE node
W20260619 17:02:21.607025 7070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:21.607172 7071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:21.607002 7073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:21.607456 7065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:21.607743 7065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:21.608914 7065 hybrid_clock.cc:648] HybridClock initialized: now 1781888541608876 us; error 51 us; skew 500 ppm
Jun 19 17:02:21 dist-test-slave-vnbj krb5kdc[6958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888541, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:21.611753 7065 init.cc:377] Logged in from keytab as kudu/127.20.231.129@KRBTEST.COM (short username kudu)
I20260619 17:02:21.612314 7065 webserver.cc:460] Webserver started at http://127.20.231.129:34487/ using document root <none> and password file <none>
I20260619 17:02:21.612525 7065 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:21.612566 7065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:21.612658 7065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:21.613799 7065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-0/data/instance:
uuid: "3a861c3e2c434f9db423f4c7536a8927"
format_stamp: "Formatted at 2026-06-19 17:02:21 on dist-test-slave-vnbj"
server_key: "7b15fa04255c94739d894df2800fabd5"
server_key_iv: "46f8d216415464e035f439bf68a2683c"
server_key_version: "encryptionkey@0"
I20260619 17:02:21.614112 7065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance:
uuid: "3a861c3e2c434f9db423f4c7536a8927"
format_stamp: "Formatted at 2026-06-19 17:02:21 on dist-test-slave-vnbj"
server_key: "7b15fa04255c94739d894df2800fabd5"
server_key_iv: "46f8d216415464e035f439bf68a2683c"
server_key_version: "encryptionkey@0"
I20260619 17:02:21.615566 7065 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 17:02:21.616433 7080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:21.616683 7065 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:21.616806 7065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-0/wal
uuid: "3a861c3e2c434f9db423f4c7536a8927"
format_stamp: "Formatted at 2026-06-19 17:02:21 on dist-test-slave-vnbj"
server_key: "7b15fa04255c94739d894df2800fabd5"
server_key_iv: "46f8d216415464e035f439bf68a2683c"
server_key_version: "encryptionkey@0"
I20260619 17:02:21.616895 7065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:21.625880 7065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:21.628379 7065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:21.628568 7065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:21.628806 7065 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:02:21.629112 7065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:02:21.629165 7065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:21.629207 7065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:02:21.629236 7065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:21.636533 7065 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.129:37153
I20260619 17:02:21.636957 7065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
I20260619 17:02:21.639703 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 7065
I20260619 17:02:21.639803 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance
I20260619 17:02:21.639998 21406 external_mini_cluster.cc:1428] Setting key 513fd02e0f76be59b7a367d8aa2581ff
I20260619 17:02:21.642868 7193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.129:37153 every 8 connection(s)
Jun 19 17:02:21 dist-test-slave-vnbj krb5kdc[6958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888541, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.231.130@KRBTEST.COM; defaulting to no policy
I20260619 17:02:21.654789 7049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:21.643165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:36577 (local address 127.20.231.190:34085)
0619 17:02:21.643215 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:21.643217 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:21.643500 (+ 283us) server_negotiation.cc:407] Connection header received
0619 17:02:21.644259 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:21.644260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:21.644299 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:21.644396 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:21.645964 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:21.646782 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:21.648560 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:21.648888 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:21.648982 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:21.651549 (+ 2567us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:21.651550 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:21.651551 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:21.651559 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:21.653111 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:21.653692 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:21.653694 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:21.653694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:21.653757 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:21.654043 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:21.654044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:21.654044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:21.654175 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:21.654255 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:21.654593 (+ 338us) server_negotiation.cc:299] Negotiation successful
0619 17:02:21.654694 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.130@KRBTEST.COM" created.
I20260619 17:02:21.655112 7197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:21.643131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:34085 (local address 127.20.231.129:36577)
0619 17:02:21.643311 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:21.643382 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:21.644134 (+ 752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:21.644516 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:21.644521 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:21.645380 (+ 859us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:21.645721 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:21.645731 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:21.646933 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:21.646934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:21.648438 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:21.648442 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:21.649007 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:21.649009 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:21.649159 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:21.649689 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:02:21.649699 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:21.651438 (+ 1739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:21.653226 (+ 1788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:21.653241 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:21.653244 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:21.653622 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:21.653835 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:21.653837 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:21.653837 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:21.653966 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:21.654246 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:21.654253 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:21.654448 (+ 195us) client_negotiation.cc:769] Sending connection context
0619 17:02:21.654572 (+ 124us) client_negotiation.cc:240] Negotiation successful
0619 17:02:21.654748 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":101,"threads_started":1}
I20260619 17:02:21.655717 7194 heartbeater.cc:347] Connected to a master server at 127.20.231.190:34085
I20260619 17:02:21.655848 7194 heartbeater.cc:464] Registering TS with master...
I20260619 17:02:21.656059 7194 heartbeater.cc:511] Master 127.20.231.190:34085 requested a full tablet report, sending...
I20260619 17:02:21.656634 7003 ts_manager.cc:194] Registered new tserver with Master: 3a861c3e2c434f9db423f4c7536a8927 (127.20.231.129:37153)
I20260619 17:02:21.657616 7003 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.129@KRBTEST.COM'} at 127.20.231.129:36577
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab.
Entry for principal kudu/127.20.231.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab.
I20260619 17:02:21.674194 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.130:0
--local_ip_for_outbound_sockets=127.20.231.130
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:34085
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:44643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:02:21.758716 7199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:21.759086 7199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:21.759117 7199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:21.760499 7199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:21.760558 7199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:21.760589 7199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.130
I20260619 17:02:21.762141 7199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:44643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:34085
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.130
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:21.762351 7199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:21.762545 7199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:21.768639 7205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:21.768828 7207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:21.768862 7204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:21.768829 7199 server_base.cc:1034] running on GCE node
I20260619 17:02:21.769304 7199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:21.769590 7199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:21.770797 7199 hybrid_clock.cc:648] HybridClock initialized: now 1781888541770780 us; error 44 us; skew 500 ppm
Jun 19 17:02:21 dist-test-slave-vnbj krb5kdc[6958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888541, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:21.773602 7199 init.cc:377] Logged in from keytab as kudu/127.20.231.130@KRBTEST.COM (short username kudu)
I20260619 17:02:21.774197 7199 webserver.cc:460] Webserver started at http://127.20.231.130:33829/ using document root <none> and password file <none>
I20260619 17:02:21.774446 7199 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:21.774518 7199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:21.774675 7199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:21.775847 7199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-1/data/instance:
uuid: "51a261d011264fad997000f38ca4ce40"
format_stamp: "Formatted at 2026-06-19 17:02:21 on dist-test-slave-vnbj"
server_key: "3c6c183156430f7f47917e22b72be93b"
server_key_iv: "c00be5adfa6fb0974723b2d8b3457404"
server_key_version: "encryptionkey@0"
I20260619 17:02:21.776243 7199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance:
uuid: "51a261d011264fad997000f38ca4ce40"
format_stamp: "Formatted at 2026-06-19 17:02:21 on dist-test-slave-vnbj"
server_key: "3c6c183156430f7f47917e22b72be93b"
server_key_iv: "c00be5adfa6fb0974723b2d8b3457404"
server_key_version: "encryptionkey@0"
I20260619 17:02:21.777685 7199 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 17:02:21.778586 7214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:21.778810 7199 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:21.778970 7199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-1/wal
uuid: "51a261d011264fad997000f38ca4ce40"
format_stamp: "Formatted at 2026-06-19 17:02:21 on dist-test-slave-vnbj"
server_key: "3c6c183156430f7f47917e22b72be93b"
server_key_iv: "c00be5adfa6fb0974723b2d8b3457404"
server_key_version: "encryptionkey@0"
I20260619 17:02:21.779054 7199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:21.786512 7199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:21.788971 7199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:21.789147 7199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:21.789386 7199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:02:21.789726 7199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:02:21.789783 7199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:21.789824 7199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:02:21.789850 7199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:21.796499 7199 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.130:44245
I20260619 17:02:21.796547 7327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.130:44245 every 8 connection(s)
I20260619 17:02:21.796902 7199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
I20260619 17:02:21.799788 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 7199
I20260619 17:02:21.799990 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance
I20260619 17:02:21.800215 21406 external_mini_cluster.cc:1428] Setting key 1646321b7c6925556dbb54089d01c311
Jun 19 17:02:21 dist-test-slave-vnbj krb5kdc[6958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888541, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.231.131@KRBTEST.COM; defaulting to no policy
I20260619 17:02:21.809487 7049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:21.797889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:49117 (local address 127.20.231.190:34085)
0619 17:02:21.797985 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:21.797987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:21.798160 (+ 173us) server_negotiation.cc:407] Connection header received
0619 17:02:21.798808 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:21.798809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:21.798830 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:21.798930 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:21.800186 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:21.800842 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:21.802596 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:21.802979 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:21.803060 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:21.805701 (+ 2641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:21.805704 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:21.805705 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:21.805715 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:21.807751 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:21.808215 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:21.808216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:21.808217 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:21.808281 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:21.808625 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:21.808626 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:21.808626 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:21.808763 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:21.808906 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:21.809273 (+ 367us) server_negotiation.cc:299] Negotiation successful
0619 17:02:21.809379 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 17:02:21.809737 7330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:21.797768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:34085 (local address 127.20.231.130:49117)
0619 17:02:21.798018 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:21.798068 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:21.798617 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:21.799018 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:21.799022 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:21.799793 (+ 771us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:21.800059 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:21.800082 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:21.801084 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:21.801085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:21.802475 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:21.802479 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:21.803074 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:21.803075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:21.803190 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:21.803766 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:02:21.803774 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:21.805588 (+ 1814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:21.807849 (+ 2261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:21.807856 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:21.807858 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:21.808129 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:21.808397 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:21.808398 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:21.808398 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:21.808515 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:21.808912 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:21.808919 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:21.809120 (+ 201us) client_negotiation.cc:769] Sending connection context
0619 17:02:21.809250 (+ 130us) client_negotiation.cc:240] Negotiation successful
0619 17:02:21.809429 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":99,"threads_started":1}
I20260619 17:02:21.810288 7328 heartbeater.cc:347] Connected to a master server at 127.20.231.190:34085
I20260619 17:02:21.810418 7328 heartbeater.cc:464] Registering TS with master...
I20260619 17:02:21.810632 7328 heartbeater.cc:511] Master 127.20.231.190:34085 requested a full tablet report, sending...
I20260619 17:02:21.811102 7003 ts_manager.cc:194] Registered new tserver with Master: 51a261d011264fad997000f38ca4ce40 (127.20.231.130:44245)
I20260619 17:02:21.811506 7003 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.130@KRBTEST.COM'} at 127.20.231.130:49117
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab.
Entry for principal kudu/127.20.231.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab.
I20260619 17:02:21.830502 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.131:0
--local_ip_for_outbound_sockets=127.20.231.131
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:34085
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:44643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:02:21.914628 7333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:21.914870 7333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:21.914897 7333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:21.916080 7333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:21.916123 7333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:21.916150 7333 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.131
I20260619 17:02:21.917538 7333 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:44643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:34085
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.131
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:21.917732 7333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:21.917917 7333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:21.923359 7341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:21.923367 7338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:21.923508 7339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:21.923589 7333 server_base.cc:1034] running on GCE node
I20260619 17:02:21.923856 7333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:21.924131 7333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:21.925305 7333 hybrid_clock.cc:648] HybridClock initialized: now 1781888541925296 us; error 44 us; skew 500 ppm
Jun 19 17:02:21 dist-test-slave-vnbj krb5kdc[6958](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888541, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:21.932076 7333 init.cc:377] Logged in from keytab as kudu/127.20.231.131@KRBTEST.COM (short username kudu)
I20260619 17:02:21.932636 7333 webserver.cc:460] Webserver started at http://127.20.231.131:41261/ using document root <none> and password file <none>
I20260619 17:02:21.932933 7333 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:21.932987 7333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:21.933136 7333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:21.934433 7333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-2/data/instance:
uuid: "77dbd91fd46e45908c29534bb459467a"
format_stamp: "Formatted at 2026-06-19 17:02:21 on dist-test-slave-vnbj"
server_key: "dd366f892aed9f3b11ff808650c7baf1"
server_key_iv: "cbc0748fd2b01c466674a476dda341dc"
server_key_version: "encryptionkey@0"
I20260619 17:02:21.934821 7333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance:
uuid: "77dbd91fd46e45908c29534bb459467a"
format_stamp: "Formatted at 2026-06-19 17:02:21 on dist-test-slave-vnbj"
server_key: "dd366f892aed9f3b11ff808650c7baf1"
server_key_iv: "cbc0748fd2b01c466674a476dda341dc"
server_key_version: "encryptionkey@0"
I20260619 17:02:21.936444 7333 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:21.937494 7348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:21.937840 7333 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:21.937930 7333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-2/wal
uuid: "77dbd91fd46e45908c29534bb459467a"
format_stamp: "Formatted at 2026-06-19 17:02:21 on dist-test-slave-vnbj"
server_key: "dd366f892aed9f3b11ff808650c7baf1"
server_key_iv: "cbc0748fd2b01c466674a476dda341dc"
server_key_version: "encryptionkey@0"
I20260619 17:02:21.938053 7333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:21.953006 7333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:21.955497 7333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:21.955662 7333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:21.955942 7333 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:02:21.956265 7333 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:02:21.956310 7333 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:21.956333 7333 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:02:21.956346 7333 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:21.962787 7333 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.131:44605
I20260619 17:02:21.962828 7461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.131:44605 every 8 connection(s)
I20260619 17:02:21.963308 7333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
I20260619 17:02:21.966029 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 7333
I20260619 17:02:21.966156 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance
I20260619 17:02:21.966327 21406 external_mini_cluster.cc:1428] Setting key f71c45a300c7b5113bd5aaac7aed90db
Jun 19 17:02:21 dist-test-slave-vnbj krb5kdc[6958](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888541, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:02:21.973881 7049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:21.964107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:47973 (local address 127.20.231.190:34085)
0619 17:02:21.964149 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:21.964151 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:21.964383 (+ 232us) server_negotiation.cc:407] Connection header received
0619 17:02:21.965418 (+ 1035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:21.965420 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:21.965436 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:21.965495 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:21.966146 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:21.966913 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:21.968107 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:21.968346 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:21.968419 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:21.970718 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:21.970720 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:21.970740 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:21.970750 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:21.972339 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:21.972743 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:21.972745 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:21.972745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:21.972808 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:21.973099 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:21.973099 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:21.973100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:21.973220 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:21.973279 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:21.973668 (+ 389us) server_negotiation.cc:299] Negotiation successful
0619 17:02:21.973738 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 17:02:21.974033 7464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:21.964097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:34085 (local address 127.20.231.131:47973)
0619 17:02:21.964245 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:21.964281 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:21.965301 (+ 1020us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:21.965567 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:21.965571 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:21.965838 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:21.966045 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:21.966049 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:21.966989 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:21.966990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:21.968004 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:21.968006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:21.968436 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:21.968437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:21.968546 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:21.969059 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:02:21.969067 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:21.970608 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:21.972421 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:21.972427 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:21.972429 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:21.972653 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:21.972913 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:21.972914 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:21.972915 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:21.973031 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:21.973311 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:21.973319 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:21.973474 (+ 155us) client_negotiation.cc:769] Sending connection context
0619 17:02:21.973588 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 17:02:21.973730 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":84,"threads_started":1}
I20260619 17:02:21.974628 7462 heartbeater.cc:347] Connected to a master server at 127.20.231.190:34085
I20260619 17:02:21.974722 7462 heartbeater.cc:464] Registering TS with master...
I20260619 17:02:21.974973 7462 heartbeater.cc:511] Master 127.20.231.190:34085 requested a full tablet report, sending...
I20260619 17:02:21.975415 7003 ts_manager.cc:194] Registered new tserver with Master: 77dbd91fd46e45908c29534bb459467a (127.20.231.131:44605)
I20260619 17:02:21.975806 7003 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.131@KRBTEST.COM'} at 127.20.231.131:47973
I20260619 17:02:21.978119 21406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 17:02:21.986891 7049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:21.979600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55764 (local address 127.20.231.190:34085)
0619 17:02:21.979666 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:21.979668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:21.979890 (+ 222us) server_negotiation.cc:407] Connection header received
0619 17:02:21.979934 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:21.979934 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:21.979943 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:21.980018 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:21.980619 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:21.981215 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:21.982107 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:21.982354 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:21.982425 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:21.983901 (+ 1476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:21.983903 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:21.983903 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:21.983910 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:21.985406 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:21.985847 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:21.985848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:21.985849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:21.985934 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:21.986158 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:21.986159 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:21.986159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:21.986333 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:21.986432 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:21.986637 (+ 205us) server_negotiation.cc:299] Negotiation successful
0619 17:02:21.986748 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 17:02:21.989010 7003 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55764:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 17:02:21.990608 7003 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 17:02:21.997406 7475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:21.993239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52488 (local address 127.20.231.129:37153)
0619 17:02:21.993430 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:21.993436 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:21.993448 (+ 12us) server_negotiation.cc:407] Connection header received
0619 17:02:21.993492 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:21.993495 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:21.993511 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:21.993617 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:02:21.993897 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:21.994679 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:21.996198 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:21.996967 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:21.997045 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:21.997102 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:21.997262 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 17:02:21.997307 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":76,"threads_started":1}
I20260619 17:02:21.997392 7473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:21.993130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:37153 (local address 127.0.0.1:52488)
0619 17:02:21.993295 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:21.993339 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:21.993405 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:21.993709 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:21.993712 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:21.993714 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:02:21.993800 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:21.993802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:21.994791 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:21.994792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:21.996100 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:21.996103 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:21.997062 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:21.997063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:21.997144 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:21.997158 (+ 14us) client_negotiation.cc:769] Sending connection context
0619 17:02:21.997234 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 17:02:21.997298 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":69,"threads_started":1}
I20260619 17:02:21.997583 7478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:21.993553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44488 (local address 127.20.231.131:44605)
0619 17:02:21.993767 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:21.993771 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:21.993781 (+ 10us) server_negotiation.cc:407] Connection header received
0619 17:02:21.993832 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:21.993834 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:21.993849 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:21.993919 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:02:21.994115 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:21.995255 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:21.996529 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:21.997052 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:21.997122 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:21.997179 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:21.997438 (+ 259us) server_negotiation.cc:299] Negotiation successful
0619 17:02:21.997489 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":87,"threads_started":1}
I20260619 17:02:21.997725 7476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:21.993355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:44605 (local address 127.0.0.1:44488)
0619 17:02:21.993587 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:21.993597 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:21.993645 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:21.993987 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:21.993987 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:21.993992 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:02:21.994034 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:21.994036 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:21.995376 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:21.995377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:21.996418 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:21.996420 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:21.997212 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:21.997213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:21.997268 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:21.997272 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 17:02:21.997315 (+ 43us) client_negotiation.cc:240] Negotiation successful
0619 17:02:21.997358 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":36,"thread_start_us":78,"threads_started":1}
I20260619 17:02:21.998328 7127 tablet_service.cc:1467] Processing CreateTablet for tablet 146a25522afe4b7e8dda9c41f3a3d007 (DEFAULT_TABLE table=test-table [id=dc2a77b2d1e04eaf98e875ff4e524f62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 17:02:21.998333 7396 tablet_service.cc:1467] Processing CreateTablet for tablet 146a25522afe4b7e8dda9c41f3a3d007 (DEFAULT_TABLE table=test-table [id=dc2a77b2d1e04eaf98e875ff4e524f62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 17:02:21.998699 7396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 146a25522afe4b7e8dda9c41f3a3d007. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:21.998699 7127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 146a25522afe4b7e8dda9c41f3a3d007. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:22.002292 7480 tablet_bootstrap.cc:492] T 146a25522afe4b7e8dda9c41f3a3d007 P 3a861c3e2c434f9db423f4c7536a8927: Bootstrap starting.
I20260619 17:02:22.002422 7474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:21.993162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:44245 (local address 127.0.0.1:36790)
0619 17:02:21.993548 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:21.993559 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:21.993605 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:21.993987 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:21.993987 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:21.993989 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:02:21.994035 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:21.994036 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:21.995394 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:21.995396 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:21.996487 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:21.996489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:21.997459 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:21.997460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:21.997767 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:21.997773 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 17:02:22.002249 (+ 4476us) client_negotiation.cc:240] Negotiation successful
0619 17:02:22.002336 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"mutex_wait_us":28,"thread_start_us":94,"threads_started":1}
I20260619 17:02:22.002735 7477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:21.993332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36790 (local address 127.20.231.130:44245)
0619 17:02:21.993756 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:21.993762 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:21.993772 (+ 10us) server_negotiation.cc:407] Connection header received
0619 17:02:21.993813 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:21.993815 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:21.993829 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:21.993904 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:02:21.994115 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:21.995264 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:21.996583 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:21.997309 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:21.997377 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:21.997423 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:22.002604 (+ 5181us) server_negotiation.cc:299] Negotiation successful
0619 17:02:22.002670 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":86,"threads_started":1}
I20260619 17:02:22.003158 7480 tablet_bootstrap.cc:654] T 146a25522afe4b7e8dda9c41f3a3d007 P 3a861c3e2c434f9db423f4c7536a8927: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:22.003494 7262 tablet_service.cc:1467] Processing CreateTablet for tablet 146a25522afe4b7e8dda9c41f3a3d007 (DEFAULT_TABLE table=test-table [id=dc2a77b2d1e04eaf98e875ff4e524f62]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 17:02:22.003757 7480 log.cc:824] T 146a25522afe4b7e8dda9c41f3a3d007 P 3a861c3e2c434f9db423f4c7536a8927: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:22.003820 7262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 146a25522afe4b7e8dda9c41f3a3d007. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:22.004127 7479 tablet_bootstrap.cc:492] T 146a25522afe4b7e8dda9c41f3a3d007 P 77dbd91fd46e45908c29534bb459467a: Bootstrap starting.
I20260619 17:02:22.004758 7479 tablet_bootstrap.cc:654] T 146a25522afe4b7e8dda9c41f3a3d007 P 77dbd91fd46e45908c29534bb459467a: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:22.005399 7479 log.cc:824] T 146a25522afe4b7e8dda9c41f3a3d007 P 77dbd91fd46e45908c29534bb459467a: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:22.006006 7480 tablet_bootstrap.cc:492] T 146a25522afe4b7e8dda9c41f3a3d007 P 3a861c3e2c434f9db423f4c7536a8927: No bootstrap required, opened a new log
I20260619 17:02:22.006102 7480 ts_tablet_manager.cc:1397] T 146a25522afe4b7e8dda9c41f3a3d007 P 3a861c3e2c434f9db423f4c7536a8927: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260619 17:02:22.006227 7479 tablet_bootstrap.cc:492] T 146a25522afe4b7e8dda9c41f3a3d007 P 77dbd91fd46e45908c29534bb459467a: No bootstrap required, opened a new log
I20260619 17:02:22.006300 7479 ts_tablet_manager.cc:1397] T 146a25522afe4b7e8dda9c41f3a3d007 P 77dbd91fd46e45908c29534bb459467a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 17:02:22.007041 7483 tablet_bootstrap.cc:492] T 146a25522afe4b7e8dda9c41f3a3d007 P 51a261d011264fad997000f38ca4ce40: Bootstrap starting.
I20260619 17:02:22.007728 7483 tablet_bootstrap.cc:654] T 146a25522afe4b7e8dda9c41f3a3d007 P 51a261d011264fad997000f38ca4ce40: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:22.008008 7479 raft_consensus.cc:348] T 146a25522afe4b7e8dda9c41f3a3d007 P 77dbd91fd46e45908c29534bb459467a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a861c3e2c434f9db423f4c7536a8927" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 37153 } } peers { permanent_uuid: "51a261d011264fad997000f38ca4ce40" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 44245 } } peers { permanent_uuid: "77dbd91fd46e45908c29534bb459467a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 44605 } }
I20260619 17:02:22.008147 7479 raft_consensus.cc:374] T 146a25522afe4b7e8dda9c41f3a3d007 P 77dbd91fd46e45908c29534bb459467a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:22.008175 7479 raft_consensus.cc:729] T 146a25522afe4b7e8dda9c41f3a3d007 P 77dbd91fd46e45908c29534bb459467a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77dbd91fd46e45908c29534bb459467a, State: Initialized, Role: FOLLOWER
I20260619 17:02:22.008193 7480 raft_consensus.cc:348] T 146a25522afe4b7e8dda9c41f3a3d007 P 3a861c3e2c434f9db423f4c7536a8927 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a861c3e2c434f9db423f4c7536a8927" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 37153 } } peers { permanent_uuid: "51a261d011264fad997000f38ca4ce40" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 44245 } } peers { permanent_uuid: "77dbd91fd46e45908c29534bb459467a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 44605 } }
I20260619 17:02:22.008394 7480 raft_consensus.cc:374] T 146a25522afe4b7e8dda9c41f3a3d007 P 3a861c3e2c434f9db423f4c7536a8927 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:22.008363 7479 consensus_queue.cc:260] T 146a25522afe4b7e8dda9c41f3a3d007 P 77dbd91fd46e45908c29534bb459467a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a861c3e2c434f9db423f4c7536a8927" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 37153 } } peers { permanent_uuid: "51a261d011264fad997000f38ca4ce40" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 44245 } } peers { permanent_uuid: "77dbd91fd46e45908c29534bb459467a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 44605 } }
I20260619 17:02:22.008430 7480 raft_consensus.cc:729] T 146a25522afe4b7e8dda9c41f3a3d007 P 3a861c3e2c434f9db423f4c7536a8927 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a861c3e2c434f9db423f4c7536a8927, State: Initialized, Role: FOLLOWER
I20260619 17:02:22.008528 7483 log.cc:824] T 146a25522afe4b7e8dda9c41f3a3d007 P 51a261d011264fad997000f38ca4ce40: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:22.008582 7480 consensus_queue.cc:260] T 146a25522afe4b7e8dda9c41f3a3d007 P 3a861c3e2c434f9db423f4c7536a8927 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a861c3e2c434f9db423f4c7536a8927" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 37153 } } peers { permanent_uuid: "51a261d011264fad997000f38ca4ce40" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 44245 } } peers { permanent_uuid: "77dbd91fd46e45908c29534bb459467a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 44605 } }
I20260619 17:02:22.008710 7479 ts_tablet_manager.cc:1428] T 146a25522afe4b7e8dda9c41f3a3d007 P 77dbd91fd46e45908c29534bb459467a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 17:02:22.008773 7462 heartbeater.cc:503] Master 127.20.231.190:34085 was elected leader, sending a full tablet report...
I20260619 17:02:22.008850 7480 ts_tablet_manager.cc:1428] T 146a25522afe4b7e8dda9c41f3a3d007 P 3a861c3e2c434f9db423f4c7536a8927: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260619 17:02:22.008971 7194 heartbeater.cc:503] Master 127.20.231.190:34085 was elected leader, sending a full tablet report...
I20260619 17:02:22.009667 7483 tablet_bootstrap.cc:492] T 146a25522afe4b7e8dda9c41f3a3d007 P 51a261d011264fad997000f38ca4ce40: No bootstrap required, opened a new log
I20260619 17:02:22.009742 7483 ts_tablet_manager.cc:1397] T 146a25522afe4b7e8dda9c41f3a3d007 P 51a261d011264fad997000f38ca4ce40: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 17:02:22.011333 7483 raft_consensus.cc:348] T 146a25522afe4b7e8dda9c41f3a3d007 P 51a261d011264fad997000f38ca4ce40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a861c3e2c434f9db423f4c7536a8927" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 37153 } } peers { permanent_uuid: "51a261d011264fad997000f38ca4ce40" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 44245 } } peers { permanent_uuid: "77dbd91fd46e45908c29534bb459467a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 44605 } }
I20260619 17:02:22.011466 7483 raft_consensus.cc:374] T 146a25522afe4b7e8dda9c41f3a3d007 P 51a261d011264fad997000f38ca4ce40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:22.011487 7483 raft_consensus.cc:729] T 146a25522afe4b7e8dda9c41f3a3d007 P 51a261d011264fad997000f38ca4ce40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51a261d011264fad997000f38ca4ce40, State: Initialized, Role: FOLLOWER
I20260619 17:02:22.011600 7483 consensus_queue.cc:260] T 146a25522afe4b7e8dda9c41f3a3d007 P 51a261d011264fad997000f38ca4ce40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a861c3e2c434f9db423f4c7536a8927" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 37153 } } peers { permanent_uuid: "51a261d011264fad997000f38ca4ce40" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 44245 } } peers { permanent_uuid: "77dbd91fd46e45908c29534bb459467a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 44605 } }
I20260619 17:02:22.011835 7483 ts_tablet_manager.cc:1428] T 146a25522afe4b7e8dda9c41f3a3d007 P 51a261d011264fad997000f38ca4ce40: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260619 17:02:22.011886 7328 heartbeater.cc:503] Master 127.20.231.190:34085 was elected leader, sending a full tablet report...
I20260619 17:02:22.027076 7487 raft_consensus.cc:482] T 146a25522afe4b7e8dda9c41f3a3d007 P 51a261d011264fad997000f38ca4ce40 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 17:02:22.027205 7487 raft_consensus.cc:504] T 146a25522afe4b7e8dda9c41f3a3d007 P 51a261d011264fad997000f38ca4ce40 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a861c3e2c434f9db423f4c7536a8927" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 37153 } } peers { permanent_uuid: "51a261d011264fad997000f38ca4ce40" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 44245 } } peers { permanent_uuid: "77dbd91fd46e45908c29534bb459467a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 44605 } }
I20260619 17:02:22.027670 7487 leader_election.cc:288] T 146a25522afe4b7e8dda9c41f3a3d007 P 51a261d011264fad997000f38ca4ce40 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a861c3e2c434f9db423f4c7536a8927 (127.20.231.129:37153), 77dbd91fd46e45908c29534bb459467a (127.20.231.131:44605)
I20260619 17:02:22.030975 7330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:22.027888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:37153 (local address 127.20.231.130:37387)
0619 17:02:22.027944 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:22.027958 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:22.028044 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:22.028230 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:22.028231 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:22.028233 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:02:22.028354 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:22.028356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.029119 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.029120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:22.030010 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:22.030013 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.030657 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.030658 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:22.030745 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:22.030748 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 17:02:22.030822 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 17:02:22.030904 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260619 17:02:22.030977 7475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:22.027990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:37387 (local address 127.20.231.129:37153)
0619 17:02:22.028043 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:22.028046 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:22.028058 (+ 12us) server_negotiation.cc:407] Connection header received
0619 17:02:22.028129 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:22.028130 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:22.028142 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:22.028204 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:02:22.028440 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.029024 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.030086 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.030574 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.030661 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:22.030671 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:22.030832 (+ 161us) server_negotiation.cc:299] Negotiation successful
0619 17:02:22.030897 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 17:02:22.031081 7478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:22.028113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:44801 (local address 127.20.231.131:44605)
0619 17:02:22.028162 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:22.028164 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:22.028269 (+ 105us) server_negotiation.cc:407] Connection header received
0619 17:02:22.028330 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:22.028331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:22.028343 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:22.028416 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:02:22.028609 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.029116 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.030164 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.030643 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.030746 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:22.030755 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:22.030958 (+ 203us) server_negotiation.cc:299] Negotiation successful
0619 17:02:22.031003 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 17:02:22.031333 7488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:22.028000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:44605 (local address 127.20.231.130:44801)
0619 17:02:22.028198 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:22.028207 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:22.028261 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:22.028443 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:22.028444 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:22.028445 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:02:22.028510 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:22.028512 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.029202 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.029203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:22.030091 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:22.030093 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.030751 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.030752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:22.030828 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:22.030832 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 17:02:22.031174 (+ 342us) client_negotiation.cc:240] Negotiation successful
0619 17:02:22.031229 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":30,"thread_start_us":118,"threads_started":1}
I20260619 17:02:22.031323 7148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "146a25522afe4b7e8dda9c41f3a3d007" candidate_uuid: "51a261d011264fad997000f38ca4ce40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a861c3e2c434f9db423f4c7536a8927" is_pre_election: true
I20260619 17:02:22.031481 7148 raft_consensus.cc:2454] T 146a25522afe4b7e8dda9c41f3a3d007 P 3a861c3e2c434f9db423f4c7536a8927 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51a261d011264fad997000f38ca4ce40 in term 0.
I20260619 17:02:22.031701 7416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "146a25522afe4b7e8dda9c41f3a3d007" candidate_uuid: "51a261d011264fad997000f38ca4ce40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77dbd91fd46e45908c29534bb459467a" is_pre_election: true
I20260619 17:02:22.031764 7215 leader_election.cc:302] T 146a25522afe4b7e8dda9c41f3a3d007 P 51a261d011264fad997000f38ca4ce40 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a861c3e2c434f9db423f4c7536a8927, 51a261d011264fad997000f38ca4ce40; no voters:
I20260619 17:02:22.031852 7416 raft_consensus.cc:2454] T 146a25522afe4b7e8dda9c41f3a3d007 P 77dbd91fd46e45908c29534bb459467a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51a261d011264fad997000f38ca4ce40 in term 0.
I20260619 17:02:22.031919 7487 raft_consensus.cc:2781] T 146a25522afe4b7e8dda9c41f3a3d007 P 51a261d011264fad997000f38ca4ce40 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 17:02:22.031985 7487 raft_consensus.cc:482] T 146a25522afe4b7e8dda9c41f3a3d007 P 51a261d011264fad997000f38ca4ce40 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 17:02:22.032016 7487 raft_consensus.cc:3037] T 146a25522afe4b7e8dda9c41f3a3d007 P 51a261d011264fad997000f38ca4ce40 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:22.032742 7487 raft_consensus.cc:504] T 146a25522afe4b7e8dda9c41f3a3d007 P 51a261d011264fad997000f38ca4ce40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a861c3e2c434f9db423f4c7536a8927" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 37153 } } peers { permanent_uuid: "51a261d011264fad997000f38ca4ce40" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 44245 } } peers { permanent_uuid: "77dbd91fd46e45908c29534bb459467a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 44605 } }
I20260619 17:02:22.032879 7487 leader_election.cc:288] T 146a25522afe4b7e8dda9c41f3a3d007 P 51a261d011264fad997000f38ca4ce40 [CANDIDATE]: Term 1 election: Requested vote from peers 3a861c3e2c434f9db423f4c7536a8927 (127.20.231.129:37153), 77dbd91fd46e45908c29534bb459467a (127.20.231.131:44605)
I20260619 17:02:22.033069 7148 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "146a25522afe4b7e8dda9c41f3a3d007" candidate_uuid: "51a261d011264fad997000f38ca4ce40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a861c3e2c434f9db423f4c7536a8927"
I20260619 17:02:22.033069 7416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "146a25522afe4b7e8dda9c41f3a3d007" candidate_uuid: "51a261d011264fad997000f38ca4ce40" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77dbd91fd46e45908c29534bb459467a"
I20260619 17:02:22.033167 7148 raft_consensus.cc:3037] T 146a25522afe4b7e8dda9c41f3a3d007 P 3a861c3e2c434f9db423f4c7536a8927 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:22.033167 7416 raft_consensus.cc:3037] T 146a25522afe4b7e8dda9c41f3a3d007 P 77dbd91fd46e45908c29534bb459467a [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:22.034143 7416 raft_consensus.cc:2454] T 146a25522afe4b7e8dda9c41f3a3d007 P 77dbd91fd46e45908c29534bb459467a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51a261d011264fad997000f38ca4ce40 in term 1.
I20260619 17:02:22.034143 7148 raft_consensus.cc:2454] T 146a25522afe4b7e8dda9c41f3a3d007 P 3a861c3e2c434f9db423f4c7536a8927 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51a261d011264fad997000f38ca4ce40 in term 1.
I20260619 17:02:22.034376 7215 leader_election.cc:302] T 146a25522afe4b7e8dda9c41f3a3d007 P 51a261d011264fad997000f38ca4ce40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51a261d011264fad997000f38ca4ce40, 77dbd91fd46e45908c29534bb459467a; no voters:
I20260619 17:02:22.034513 7487 raft_consensus.cc:2781] T 146a25522afe4b7e8dda9c41f3a3d007 P 51a261d011264fad997000f38ca4ce40 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:02:22.034719 7487 raft_consensus.cc:686] T 146a25522afe4b7e8dda9c41f3a3d007 P 51a261d011264fad997000f38ca4ce40 [term 1 LEADER]: Becoming Leader. State: Replica: 51a261d011264fad997000f38ca4ce40, State: Running, Role: LEADER
I20260619 17:02:22.034834 7487 consensus_queue.cc:237] T 146a25522afe4b7e8dda9c41f3a3d007 P 51a261d011264fad997000f38ca4ce40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a861c3e2c434f9db423f4c7536a8927" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 37153 } } peers { permanent_uuid: "51a261d011264fad997000f38ca4ce40" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 44245 } } peers { permanent_uuid: "77dbd91fd46e45908c29534bb459467a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 44605 } }
I20260619 17:02:22.035763 7002 catalog_manager.cc:5495] T 146a25522afe4b7e8dda9c41f3a3d007 P 51a261d011264fad997000f38ca4ce40 reported cstate change: term changed from 0 to 1, leader changed from <none> to 51a261d011264fad997000f38ca4ce40 (127.20.231.130). New cstate: current_term: 1 leader_uuid: "51a261d011264fad997000f38ca4ce40" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a861c3e2c434f9db423f4c7536a8927" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 37153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51a261d011264fad997000f38ca4ce40" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 44245 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "77dbd91fd46e45908c29534bb459467a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 44605 } health_report { overall_health: UNKNOWN } } }
I20260619 17:02:22.044986 7477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:22.042538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36806 (local address 127.20.231.130:44245)
0619 17:02:22.042607 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:22.042609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:22.042616 (+ 7us) server_negotiation.cc:407] Connection header received
0619 17:02:22.042652 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:22.042653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:22.042671 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:22.042748 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 17:02:22.042920 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.043422 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.044206 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.044414 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.044489 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:22.044631 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 17:02:22.044727 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 17:02:22.044834 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:22.044864 (+ 30us) server_negotiation.cc:299] Negotiation successful
0619 17:02:22.044874 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 17:02:22.047266 7148 raft_consensus.cc:1261] T 146a25522afe4b7e8dda9c41f3a3d007 P 3a861c3e2c434f9db423f4c7536a8927 [term 1 FOLLOWER]: Refusing update from remote peer 51a261d011264fad997000f38ca4ce40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260619 17:02:22.047451 7329 tablet.cc:2369] T 146a25522afe4b7e8dda9c41f3a3d007 P 51a261d011264fad997000f38ca4ce40: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 17:02:22.047678 7416 raft_consensus.cc:1261] T 146a25522afe4b7e8dda9c41f3a3d007 P 77dbd91fd46e45908c29534bb459467a [term 1 FOLLOWER]: Refusing update from remote peer 51a261d011264fad997000f38ca4ce40: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 17:02:22.047688 7489 consensus_queue.cc:1035] T 146a25522afe4b7e8dda9c41f3a3d007 P 51a261d011264fad997000f38ca4ce40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a861c3e2c434f9db423f4c7536a8927" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 37153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 17:02:22.047951 7487 consensus_queue.cc:1035] T 146a25522afe4b7e8dda9c41f3a3d007 P 51a261d011264fad997000f38ca4ce40 [LEADER]: Connected to new peer: Peer: permanent_uuid: "77dbd91fd46e45908c29534bb459467a" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 44605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 17:02:22.050994 7493 mvcc.cc:205] Tried to move back new op lower bound from 7298615468221739008 to 7298615468175577088. Current Snapshot: MvccSnapshot[applied={T|T < 7298615468221739008}]
I20260619 17:02:22.053314 7003 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55764:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 17:02:22.053404 7003 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55764:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 17:02:22.054275 7003 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 69e533a86cf3419e9fc6199256b2bcae: Sending DeleteTablet for 3 replicas of tablet 146a25522afe4b7e8dda9c41f3a3d007
I20260619 17:02:22.054670 7127 tablet_service.cc:1514] Processing DeleteTablet for tablet 146a25522afe4b7e8dda9c41f3a3d007 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:02:22 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:52488
I20260619 17:02:22.054674 7262 tablet_service.cc:1514] Processing DeleteTablet for tablet 146a25522afe4b7e8dda9c41f3a3d007 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:02:22 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:36790
I20260619 17:02:22.054962 7396 tablet_service.cc:1514] Processing DeleteTablet for tablet 146a25522afe4b7e8dda9c41f3a3d007 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:02:22 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:44488
I20260619 17:02:22.055187 7504 tablet_replica.cc:330] T 146a25522afe4b7e8dda9c41f3a3d007 P 77dbd91fd46e45908c29534bb459467a: stopping tablet replica
I20260619 17:02:22.055203 7503 tablet_replica.cc:330] T 146a25522afe4b7e8dda9c41f3a3d007 P 51a261d011264fad997000f38ca4ce40: stopping tablet replica
I20260619 17:02:22.055202 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 7065
I20260619 17:02:22.055296 7503 raft_consensus.cc:2229] T 146a25522afe4b7e8dda9c41f3a3d007 P 51a261d011264fad997000f38ca4ce40 [term 1 LEADER]: Raft consensus shutting down.
I20260619 17:02:22.055454 7503 raft_consensus.cc:2258] T 146a25522afe4b7e8dda9c41f3a3d007 P 51a261d011264fad997000f38ca4ce40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 17:02:22.055666 7504 raft_consensus.cc:2229] T 146a25522afe4b7e8dda9c41f3a3d007 P 77dbd91fd46e45908c29534bb459467a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 17:02:22.055959 7503 ts_tablet_manager.cc:1905] T 146a25522afe4b7e8dda9c41f3a3d007 P 51a261d011264fad997000f38ca4ce40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 17:02:22.055855 7504 raft_consensus.cc:2258] T 146a25522afe4b7e8dda9c41f3a3d007 P 77dbd91fd46e45908c29534bb459467a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 17:02:22.056679 7504 ts_tablet_manager.cc:1905] T 146a25522afe4b7e8dda9c41f3a3d007 P 77dbd91fd46e45908c29534bb459467a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 17:02:22.057854 7503 ts_tablet_manager.cc:1918] T 146a25522afe4b7e8dda9c41f3a3d007 P 51a261d011264fad997000f38ca4ce40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 17:02:22.057934 7503 log.cc:1196] T 146a25522afe4b7e8dda9c41f3a3d007 P 51a261d011264fad997000f38ca4ce40: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-1/wal/wals/146a25522afe4b7e8dda9c41f3a3d007
I20260619 17:02:22.058218 7503 ts_tablet_manager.cc:1939] T 146a25522afe4b7e8dda9c41f3a3d007 P 51a261d011264fad997000f38ca4ce40: Deleting consensus metadata
I20260619 17:02:22.058678 7504 ts_tablet_manager.cc:1918] T 146a25522afe4b7e8dda9c41f3a3d007 P 77dbd91fd46e45908c29534bb459467a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 17:02:22.058810 7504 log.cc:1196] T 146a25522afe4b7e8dda9c41f3a3d007 P 77dbd91fd46e45908c29534bb459467a: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781888301345144-21406-0/minicluster-data/ts-2/wal/wals/146a25522afe4b7e8dda9c41f3a3d007
I20260619 17:02:22.058817 6988 catalog_manager.cc:4841] TS 51a261d011264fad997000f38ca4ce40 (127.20.231.130:44245): tablet 146a25522afe4b7e8dda9c41f3a3d007 (table test-table [id=dc2a77b2d1e04eaf98e875ff4e524f62]) successfully deleted
I20260619 17:02:22.059175 7504 ts_tablet_manager.cc:1939] T 146a25522afe4b7e8dda9c41f3a3d007 P 77dbd91fd46e45908c29534bb459467a: Deleting consensus metadata
I20260619 17:02:22.059597 6987 catalog_manager.cc:4841] TS 77dbd91fd46e45908c29534bb459467a (127.20.231.131:44605): tablet 146a25522afe4b7e8dda9c41f3a3d007 (table test-table [id=dc2a77b2d1e04eaf98e875ff4e524f62]) successfully deleted
W20260619 17:02:22.063652 6987 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.231.129:37153 (error 108)
I20260619 17:02:22.063792 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 7199
I20260619 17:02:22.064637 7474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:22.064470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:37153 (local address 127.0.0.1:52500)
0619 17:02:22.064539 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:22.064592 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.129:37153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260619 17:02:22.067410 6987 catalog_manager.cc:4584] TS 3a861c3e2c434f9db423f4c7536a8927 (127.20.231.129:37153): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 146a25522afe4b7e8dda9c41f3a3d007: Network error: Client connection negotiation failed: client connection to 127.20.231.129:37153: connect: Connection refused (error 111)
I20260619 17:02:22.068553 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 7333
I20260619 17:02:22.073802 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 6971
2026-06-19T17:02:22Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2013 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-06-19T17:02:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T17:02:22Z Disabled control of system clock
I20260619 17:02:22.103303 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:45497
--webserver_interface=127.20.231.190
--webserver_port=0
--builtin_ntp_servers=127.20.231.148:42327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:45497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 17:02:22.188928 7509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:22.189163 7509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:22.189185 7509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:22.190346 7509 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:02:22.190388 7509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:22.190402 7509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:22.190413 7509 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:02:22.190425 7509 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:02:22.192528 7509 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:42327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:45497
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:45497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:22.192920 7509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:22.193233 7509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:22.198611 7514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:22.198753 7515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:22.198753 7517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:22.198761 7509 server_base.cc:1034] running on GCE node
I20260619 17:02:22.199222 7509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:22.199570 7509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:22.200770 7509 hybrid_clock.cc:648] HybridClock initialized: now 1781888542200750 us; error 52 us; skew 500 ppm
I20260619 17:02:22.202237 7509 webserver.cc:460] Webserver started at http://127.20.231.190:34367/ using document root <none> and password file <none>
I20260619 17:02:22.202545 7509 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:22.202626 7509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:22.202912 7509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:22.204166 7509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "0d15436f445f4ce390476382a8eafe3d"
format_stamp: "Formatted at 2026-06-19 17:02:22 on dist-test-slave-vnbj"
server_key: "da8cbd745d40013a6703011e3dd89c61"
server_key_iv: "e60a76f42e1868f29b6f0a8e3b65a353"
server_key_version: "encryptionkey@0"
I20260619 17:02:22.204555 7509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "0d15436f445f4ce390476382a8eafe3d"
format_stamp: "Formatted at 2026-06-19 17:02:22 on dist-test-slave-vnbj"
server_key: "da8cbd745d40013a6703011e3dd89c61"
server_key_iv: "e60a76f42e1868f29b6f0a8e3b65a353"
server_key_version: "encryptionkey@0"
I20260619 17:02:22.206073 7509 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 17:02:22.207002 7523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:22.207216 7509 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:22.207331 7509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "0d15436f445f4ce390476382a8eafe3d"
format_stamp: "Formatted at 2026-06-19 17:02:22 on dist-test-slave-vnbj"
server_key: "da8cbd745d40013a6703011e3dd89c61"
server_key_iv: "e60a76f42e1868f29b6f0a8e3b65a353"
server_key_version: "encryptionkey@0"
I20260619 17:02:22.207425 7509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:22.240587 7509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:22.240921 7509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:22.241047 7509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:22.245671 7509 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:45497
I20260619 17:02:22.245754 7575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:45497 every 8 connection(s)
I20260619 17:02:22.246150 7509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 17:02:22.247287 7576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:22.248260 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 7509
I20260619 17:02:22.248375 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 17:02:22.248651 21406 external_mini_cluster.cc:1428] Setting key f0a6975e776a2b104d292b3417f2b64b
I20260619 17:02:22.250555 7576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d15436f445f4ce390476382a8eafe3d: Bootstrap starting.
I20260619 17:02:22.251336 7576 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d15436f445f4ce390476382a8eafe3d: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:22.251662 7576 log.cc:824] T 00000000000000000000000000000000 P 0d15436f445f4ce390476382a8eafe3d: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:22.252477 7576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d15436f445f4ce390476382a8eafe3d: No bootstrap required, opened a new log
I20260619 17:02:22.255106 7576 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d15436f445f4ce390476382a8eafe3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d15436f445f4ce390476382a8eafe3d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 45497 } }
I20260619 17:02:22.255348 7576 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d15436f445f4ce390476382a8eafe3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:22.255395 7576 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d15436f445f4ce390476382a8eafe3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d15436f445f4ce390476382a8eafe3d, State: Initialized, Role: FOLLOWER
I20260619 17:02:22.255563 7576 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d15436f445f4ce390476382a8eafe3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d15436f445f4ce390476382a8eafe3d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 45497 } }
I20260619 17:02:22.255666 7576 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d15436f445f4ce390476382a8eafe3d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 17:02:22.255704 7576 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d15436f445f4ce390476382a8eafe3d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 17:02:22.255731 7579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:22.249328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46542 (local address 127.20.231.190:45497)
0619 17:02:22.249606 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:22.249616 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:22.249651 (+ 35us) server_negotiation.cc:407] Connection header received
0619 17:02:22.250217 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:22.250227 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:22.250315 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:22.250484 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:22.250727 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.252204 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.253111 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.253769 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.253900 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:22.254802 (+ 902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:22.254807 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:22.254808 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:22.255067 (+ 259us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:22.255113 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:22.255131 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:22.255211 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:22.255334 (+ 123us) server_negotiation.cc:299] Negotiation successful
0619 17:02:22.255472 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":137,"threads_started":1}
I20260619 17:02:22.255759 7576 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d15436f445f4ce390476382a8eafe3d [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:22.256594 7576 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d15436f445f4ce390476382a8eafe3d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d15436f445f4ce390476382a8eafe3d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 45497 } }
I20260619 17:02:22.256690 7576 leader_election.cc:302] T 00000000000000000000000000000000 P 0d15436f445f4ce390476382a8eafe3d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d15436f445f4ce390476382a8eafe3d; no voters:
I20260619 17:02:22.256829 7576 leader_election.cc:288] T 00000000000000000000000000000000 P 0d15436f445f4ce390476382a8eafe3d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 17:02:22.256928 7581 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d15436f445f4ce390476382a8eafe3d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:02:22.257108 7581 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d15436f445f4ce390476382a8eafe3d [term 1 LEADER]: Becoming Leader. State: Replica: 0d15436f445f4ce390476382a8eafe3d, State: Running, Role: LEADER
I20260619 17:02:22.257200 7581 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d15436f445f4ce390476382a8eafe3d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d15436f445f4ce390476382a8eafe3d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 45497 } }
I20260619 17:02:22.257203 7576 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d15436f445f4ce390476382a8eafe3d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:02:22.257732 7583 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d15436f445f4ce390476382a8eafe3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d15436f445f4ce390476382a8eafe3d. Latest consensus state: current_term: 1 leader_uuid: "0d15436f445f4ce390476382a8eafe3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d15436f445f4ce390476382a8eafe3d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 45497 } } }
I20260619 17:02:22.257769 7582 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d15436f445f4ce390476382a8eafe3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d15436f445f4ce390476382a8eafe3d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d15436f445f4ce390476382a8eafe3d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 45497 } } }
I20260619 17:02:22.257956 7583 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d15436f445f4ce390476382a8eafe3d [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:22.257972 7582 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d15436f445f4ce390476382a8eafe3d [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:22.258435 7590 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 17:02:22.259073 7590 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 17:02:22.260649 7590 catalog_manager.cc:1347] Generated new cluster ID: 0f06485fdcb04c47885daf1249a10c29
I20260619 17:02:22.260703 7590 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 17:02:22.269107 7590 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 17:02:22.269699 7590 catalog_manager.cc:1504] Loading token signing keys...
I20260619 17:02:22.272841 7590 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d15436f445f4ce390476382a8eafe3d: Generated new TSK 0
I20260619 17:02:22.273049 7590 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 17:02:22.276321 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.129:0
--local_ip_for_outbound_sockets=127.20.231.129
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:45497
--builtin_ntp_servers=127.20.231.148:42327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 17:02:22.365526 7600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:22.365738 7600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:22.365761 7600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:22.367030 7600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:22.367070 7600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:22.367097 7600 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.129
I20260619 17:02:22.368487 7600 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:42327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:45497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.129
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:22.368753 7600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:22.369081 7600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:22.374333 7606 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:22.374323 7608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:22.374323 7605 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:22.374789 7600 server_base.cc:1034] running on GCE node
I20260619 17:02:22.375039 7600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:22.375295 7600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:22.376441 7600 hybrid_clock.cc:648] HybridClock initialized: now 1781888542376427 us; error 30 us; skew 500 ppm
I20260619 17:02:22.377693 7600 webserver.cc:460] Webserver started at http://127.20.231.129:46587/ using document root <none> and password file <none>
I20260619 17:02:22.377987 7600 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:22.378037 7600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:22.378182 7600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:22.379343 7600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-0/data/instance:
uuid: "0fcb80ea14334ec19b5043e4c6b66bd1"
format_stamp: "Formatted at 2026-06-19 17:02:22 on dist-test-slave-vnbj"
server_key: "4859f35b79ca60a19dc7ebf20bfa88d4"
server_key_iv: "9401014b42477c71368c1e529ca9a923"
server_key_version: "encryptionkey@0"
I20260619 17:02:22.379686 7600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance:
uuid: "0fcb80ea14334ec19b5043e4c6b66bd1"
format_stamp: "Formatted at 2026-06-19 17:02:22 on dist-test-slave-vnbj"
server_key: "4859f35b79ca60a19dc7ebf20bfa88d4"
server_key_iv: "9401014b42477c71368c1e529ca9a923"
server_key_version: "encryptionkey@0"
I20260619 17:02:22.381071 7600 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 17:02:22.382093 7614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:22.382323 7600 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:22.382429 7600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-0/wal
uuid: "0fcb80ea14334ec19b5043e4c6b66bd1"
format_stamp: "Formatted at 2026-06-19 17:02:22 on dist-test-slave-vnbj"
server_key: "4859f35b79ca60a19dc7ebf20bfa88d4"
server_key_iv: "9401014b42477c71368c1e529ca9a923"
server_key_version: "encryptionkey@0"
I20260619 17:02:22.382524 7600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:22.390201 7600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:22.390525 7600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:22.390686 7600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:22.391004 7600 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:02:22.391420 7600 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:02:22.391474 7600 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:22.391533 7600 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:02:22.391557 7600 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:22.397750 7600 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.129:35335
I20260619 17:02:22.397789 7727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.129:35335 every 8 connection(s)
I20260619 17:02:22.398190 7600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
I20260619 17:02:22.401928 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 7600
I20260619 17:02:22.402035 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance
I20260619 17:02:22.402227 21406 external_mini_cluster.cc:1428] Setting key 6273d97153e04a8bb7edc1d821d0a2fe
I20260619 17:02:22.403406 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.130:0
--local_ip_for_outbound_sockets=127.20.231.130
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:45497
--builtin_ntp_servers=127.20.231.148:42327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 17:02:22.404233 7579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:22.399006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:59551 (local address 127.20.231.190:45497)
0619 17:02:22.399070 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:22.399072 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:22.399360 (+ 288us) server_negotiation.cc:407] Connection header received
0619 17:02:22.399900 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:22.399901 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:22.399910 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:22.399963 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:22.400677 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.401172 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.402593 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.402917 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.402984 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:22.403944 (+ 960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:22.403946 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:22.403946 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:22.403951 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:22.403967 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:22.403971 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:22.404025 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:22.404121 (+ 96us) server_negotiation.cc:299] Negotiation successful
0619 17:02:22.404155 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 17:02:22.404462 7730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:22.399005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:45497 (local address 127.20.231.129:59551)
0619 17:02:22.399207 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:22.399243 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:22.399749 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:22.400036 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:22.400039 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:22.400048 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:22.400605 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:22.400610 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.401328 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.401329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:22.402510 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:22.402513 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.402992 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.402993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:22.403139 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:22.403723 (+ 584us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 17:02:22.403723 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:22.403752 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 17:02:22.403775 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:22.404028 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:22.404034 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:22.404038 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 17:02:22.404125 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 17:02:22.404231 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":101,"threads_started":1}
I20260619 17:02:22.405020 7728 heartbeater.cc:347] Connected to a master server at 127.20.231.190:45497
I20260619 17:02:22.405118 7728 heartbeater.cc:464] Registering TS with master...
I20260619 17:02:22.405326 7728 heartbeater.cc:511] Master 127.20.231.190:45497 requested a full tablet report, sending...
I20260619 17:02:22.405817 7540 ts_manager.cc:194] Registered new tserver with Master: 0fcb80ea14334ec19b5043e4c6b66bd1 (127.20.231.129:35335)
I20260619 17:02:22.406467 7540 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.231.129:59551
W20260619 17:02:22.486497 7731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:22.486742 7731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:22.486780 7731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:22.488070 7731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:22.488145 7731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:22.488190 7731 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.130
I20260619 17:02:22.489645 7731 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:42327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:45497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7731
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.130
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:22.489948 7731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:22.490259 7731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:22.496039 7737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:22.496116 7731 server_base.cc:1034] running on GCE node
W20260619 17:02:22.496075 7739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:22.496075 7736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:22.496600 7731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:22.496881 7731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:22.498068 7731 hybrid_clock.cc:648] HybridClock initialized: now 1781888542498065 us; error 43 us; skew 500 ppm
I20260619 17:02:22.499630 7731 webserver.cc:460] Webserver started at http://127.20.231.130:34265/ using document root <none> and password file <none>
I20260619 17:02:22.499913 7731 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:22.499991 7731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:22.500144 7731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:22.501332 7731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-1/data/instance:
uuid: "cc38513f91d74c76a1260bde3b535c63"
format_stamp: "Formatted at 2026-06-19 17:02:22 on dist-test-slave-vnbj"
server_key: "e9c8925b5cb42f1b33303d89d051f919"
server_key_iv: "e701a709afbf8e9672105b8b863d93d7"
server_key_version: "encryptionkey@0"
I20260619 17:02:22.501698 7731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance:
uuid: "cc38513f91d74c76a1260bde3b535c63"
format_stamp: "Formatted at 2026-06-19 17:02:22 on dist-test-slave-vnbj"
server_key: "e9c8925b5cb42f1b33303d89d051f919"
server_key_iv: "e701a709afbf8e9672105b8b863d93d7"
server_key_version: "encryptionkey@0"
I20260619 17:02:22.503286 7731 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 17:02:22.504249 7745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:22.504513 7731 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:22.504627 7731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-1/wal
uuid: "cc38513f91d74c76a1260bde3b535c63"
format_stamp: "Formatted at 2026-06-19 17:02:22 on dist-test-slave-vnbj"
server_key: "e9c8925b5cb42f1b33303d89d051f919"
server_key_iv: "e701a709afbf8e9672105b8b863d93d7"
server_key_version: "encryptionkey@0"
I20260619 17:02:22.504709 7731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:22.524235 7731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:22.524600 7731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:22.524765 7731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:22.525048 7731 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:02:22.525437 7731 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:02:22.525503 7731 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:22.525542 7731 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:02:22.525569 7731 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:22.531806 7731 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.130:34243
I20260619 17:02:22.531852 7858 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.130:34243 every 8 connection(s)
I20260619 17:02:22.532214 7731 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
I20260619 17:02:22.538318 7579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:22.533035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:58413 (local address 127.20.231.190:45497)
0619 17:02:22.533104 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:22.533107 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:22.533412 (+ 305us) server_negotiation.cc:407] Connection header received
0619 17:02:22.533962 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:22.533963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:22.533973 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:22.534026 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:22.534746 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.535286 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.536390 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.536581 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.536649 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:22.537744 (+ 1095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:22.537745 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:22.537745 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:22.537751 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:22.537775 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:22.537779 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:22.537834 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:22.538222 (+ 388us) server_negotiation.cc:299] Negotiation successful
0619 17:02:22.538247 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 17:02:22.538573 7861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:22.533033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:45497 (local address 127.20.231.130:58413)
0619 17:02:22.533276 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:22.533316 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:22.533872 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:22.534080 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:22.534083 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:22.534092 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:22.534599 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:22.534604 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.535404 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.535405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:22.536285 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:22.536289 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.536695 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.536696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:22.536888 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:22.537575 (+ 687us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 17:02:22.537576 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:22.537606 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 17:02:22.537630 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:22.538108 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:22.538115 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:22.538118 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 17:02:22.538194 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 17:02:22.538291 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":86,"threads_started":1}
I20260619 17:02:22.539135 7859 heartbeater.cc:347] Connected to a master server at 127.20.231.190:45497
I20260619 17:02:22.539247 7859 heartbeater.cc:464] Registering TS with master...
I20260619 17:02:22.539443 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 7731
I20260619 17:02:22.539471 7859 heartbeater.cc:511] Master 127.20.231.190:45497 requested a full tablet report, sending...
I20260619 17:02:22.539556 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance
I20260619 17:02:22.539799 21406 external_mini_cluster.cc:1428] Setting key c3e2b871769e0531191a17a3fa7bd333
I20260619 17:02:22.539983 7540 ts_manager.cc:194] Registered new tserver with Master: cc38513f91d74c76a1260bde3b535c63 (127.20.231.130:34243)
I20260619 17:02:22.540637 7540 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.231.130:58413
I20260619 17:02:22.541136 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.131:0
--local_ip_for_outbound_sockets=127.20.231.131
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:45497
--builtin_ntp_servers=127.20.231.148:42327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 17:02:22.624330 7862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:22.624575 7862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:22.624614 7862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:22.625813 7862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:22.625880 7862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:22.625936 7862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.131
I20260619 17:02:22.627470 7862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:42327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:45497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.131
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:22.627754 7862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:22.628070 7862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:22.633671 7868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:22.633666 7867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:22.633671 7870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:22.634238 7862 server_base.cc:1034] running on GCE node
I20260619 17:02:22.634514 7862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:22.634788 7862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:22.635979 7862 hybrid_clock.cc:648] HybridClock initialized: now 1781888542635956 us; error 48 us; skew 500 ppm
I20260619 17:02:22.637705 7862 webserver.cc:460] Webserver started at http://127.20.231.131:44851/ using document root <none> and password file <none>
I20260619 17:02:22.638047 7862 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:22.638127 7862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:22.638275 7862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:22.640059 7862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-2/data/instance:
uuid: "b80d695df1934011ad0ea7369d5010ef"
format_stamp: "Formatted at 2026-06-19 17:02:22 on dist-test-slave-vnbj"
server_key: "9edaf62388b0d56c0b6d1f65501de64b"
server_key_iv: "fb3882042690075824f13ea5e2861fca"
server_key_version: "encryptionkey@0"
I20260619 17:02:22.640551 7862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance:
uuid: "b80d695df1934011ad0ea7369d5010ef"
format_stamp: "Formatted at 2026-06-19 17:02:22 on dist-test-slave-vnbj"
server_key: "9edaf62388b0d56c0b6d1f65501de64b"
server_key_iv: "fb3882042690075824f13ea5e2861fca"
server_key_version: "encryptionkey@0"
I20260619 17:02:22.642534 7862 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260619 17:02:22.643548 7876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:22.643745 7862 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 17:02:22.643850 7862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-2/wal
uuid: "b80d695df1934011ad0ea7369d5010ef"
format_stamp: "Formatted at 2026-06-19 17:02:22 on dist-test-slave-vnbj"
server_key: "9edaf62388b0d56c0b6d1f65501de64b"
server_key_iv: "fb3882042690075824f13ea5e2861fca"
server_key_version: "encryptionkey@0"
I20260619 17:02:22.643941 7862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:22.657523 7862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:22.657876 7862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:22.658056 7862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:22.658331 7862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:02:22.658739 7862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:02:22.658807 7862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:22.658842 7862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:02:22.658927 7862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:22.665169 7862 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.131:38193
I20260619 17:02:22.665227 7989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.131:38193 every 8 connection(s)
I20260619 17:02:22.665603 7862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
I20260619 17:02:22.666190 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 7862
I20260619 17:02:22.666322 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance
I20260619 17:02:22.666510 21406 external_mini_cluster.cc:1428] Setting key b4f0dc09a29aff462147354f7a37cc61
I20260619 17:02:22.671437 7579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:22.666765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:57677 (local address 127.20.231.190:45497)
0619 17:02:22.666813 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:22.666815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:22.666826 (+ 11us) server_negotiation.cc:407] Connection header received
0619 17:02:22.667496 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:22.667497 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:22.667509 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:22.667562 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:22.668268 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.668770 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.669803 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.670015 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.670102 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:22.671127 (+ 1025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:22.671129 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:22.671129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:22.671134 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:22.671149 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:22.671153 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:22.671203 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:22.671323 (+ 120us) server_negotiation.cc:299] Negotiation successful
0619 17:02:22.671351 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 17:02:22.671711 7992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:22.666439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:45497 (local address 127.20.231.131:57677)
0619 17:02:22.666656 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:22.666728 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:22.667383 (+ 655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:22.667632 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:22.667636 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:22.667652 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:22.668154 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:22.668159 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.668868 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.668869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:22.669686 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:22.669688 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.670112 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.670113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:22.670263 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:22.670970 (+ 707us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 17:02:22.670970 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:22.671000 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 17:02:22.671025 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:22.671210 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:22.671215 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:22.671218 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 17:02:22.671309 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 17:02:22.671419 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":78,"threads_started":1}
I20260619 17:02:22.672263 7990 heartbeater.cc:347] Connected to a master server at 127.20.231.190:45497
I20260619 17:02:22.672354 7990 heartbeater.cc:464] Registering TS with master...
I20260619 17:02:22.672565 7990 heartbeater.cc:511] Master 127.20.231.190:45497 requested a full tablet report, sending...
I20260619 17:02:22.673064 7540 ts_manager.cc:194] Registered new tserver with Master: b80d695df1934011ad0ea7369d5010ef (127.20.231.131:38193)
I20260619 17:02:22.673544 7540 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.231.131:57677
I20260619 17:02:22.678489 21406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 17:02:22.683609 7579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:22.680119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46548 (local address 127.20.231.190:45497)
0619 17:02:22.680228 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:22.680229 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:22.680343 (+ 114us) server_negotiation.cc:407] Connection header received
0619 17:02:22.680369 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:22.680370 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:22.680380 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:22.680437 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:22.680637 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.681219 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.682054 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.682246 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.682313 (+ 67us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:22.683354 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:22.683355 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:22.683355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:22.683360 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:22.683374 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:22.683377 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:22.683468 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:22.683535 (+ 67us) server_negotiation.cc:299] Negotiation successful
0619 17:02:22.683546 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260619 17:02:22.685456 7540 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46548:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 17:02:22.687640 7540 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 17:02:22.694840 8002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:22.690790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42036 (local address 127.20.231.130:34243)
0619 17:02:22.690997 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:22.691001 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:22.691013 (+ 12us) server_negotiation.cc:407] Connection header received
0619 17:02:22.691058 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:22.691060 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:22.691077 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:22.691202 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:02:22.691301 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.692366 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.693604 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.694345 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.694427 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:22.694527 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:22.694693 (+ 166us) server_negotiation.cc:299] Negotiation successful
0619 17:02:22.694747 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":112,"threads_started":1}
I20260619 17:02:22.694875 7999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:22.690374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:34243 (local address 127.0.0.1:42036)
0619 17:02:22.690555 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:22.690567 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:22.690620 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:22.691202 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:22.691203 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:22.691204 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:02:22.691240 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:22.691241 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.692455 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.692456 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:22.693499 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:22.693501 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.694446 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.694447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:22.694562 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:22.694584 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 17:02:22.694666 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 17:02:22.694742 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":122,"threads_started":1}
I20260619 17:02:22.694958 8003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:22.691020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43646 (local address 127.20.231.131:38193)
0619 17:02:22.691411 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:22.691415 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:22.691426 (+ 11us) server_negotiation.cc:407] Connection header received
0619 17:02:22.691467 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:22.691469 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:22.691485 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:22.691565 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:02:22.691980 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.692858 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.693958 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.694520 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.694613 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:22.694703 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:22.694785 (+ 82us) server_negotiation.cc:299] Negotiation successful
0619 17:02:22.694826 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":105,"threads_started":1}
I20260619 17:02:22.695420 8000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:22.690630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:35335 (local address 127.0.0.1:37054)
0619 17:02:22.690751 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:22.690761 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:22.690804 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:22.691094 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:22.691098 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:22.691100 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:02:22.691171 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:22.691173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.692192 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.692192 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:22.693495 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:22.693499 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.695160 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.695161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:22.695209 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:22.695213 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 17:02:22.695285 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 17:02:22.695330 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":3,"thread_start_us":74,"threads_started":1}
I20260619 17:02:22.695502 7998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:22.690367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:38193 (local address 127.0.0.1:43646)
0619 17:02:22.690486 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:22.690528 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:22.690594 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:22.691638 (+ 1044us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:22.691639 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:22.691641 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:02:22.691684 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:22.691686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.692959 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.692960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:22.693863 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:22.693866 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.694612 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.694613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:22.694694 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:22.694699 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 17:02:22.694758 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 17:02:22.694797 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":1,"thread_start_us":67,"threads_started":1}
I20260619 17:02:22.695781 7793 tablet_service.cc:1467] Processing CreateTablet for tablet d4d4f250a3d845bd8d97f86c5fc31d10 (DEFAULT_TABLE table=test-table [id=b80b68753b1340edb6173f8815bf7733]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 17:02:22.696189 7793 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4d4f250a3d845bd8d97f86c5fc31d10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:22.696367 7924 tablet_service.cc:1467] Processing CreateTablet for tablet d4d4f250a3d845bd8d97f86c5fc31d10 (DEFAULT_TABLE table=test-table [id=b80b68753b1340edb6173f8815bf7733]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 17:02:22.696727 7924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4d4f250a3d845bd8d97f86c5fc31d10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:22.699268 8001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:22.690728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37054 (local address 127.20.231.129:35335)
0619 17:02:22.690926 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:22.690931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:22.690943 (+ 12us) server_negotiation.cc:407] Connection header received
0619 17:02:22.690987 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:22.690989 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:22.691005 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:22.691087 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:02:22.691259 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.692094 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.693605 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:22.694345 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:22.694427 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:22.694527 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:22.699065 (+ 4538us) server_negotiation.cc:299] Negotiation successful
0619 17:02:22.699132 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":108,"threads_started":1}
I20260619 17:02:22.700048 8004 tablet_bootstrap.cc:492] T d4d4f250a3d845bd8d97f86c5fc31d10 P cc38513f91d74c76a1260bde3b535c63: Bootstrap starting.
I20260619 17:02:22.700100 7662 tablet_service.cc:1467] Processing CreateTablet for tablet d4d4f250a3d845bd8d97f86c5fc31d10 (DEFAULT_TABLE table=test-table [id=b80b68753b1340edb6173f8815bf7733]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 17:02:22.700424 8005 tablet_bootstrap.cc:492] T d4d4f250a3d845bd8d97f86c5fc31d10 P b80d695df1934011ad0ea7369d5010ef: Bootstrap starting.
I20260619 17:02:22.700453 7662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d4d4f250a3d845bd8d97f86c5fc31d10. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:22.701006 8004 tablet_bootstrap.cc:654] T d4d4f250a3d845bd8d97f86c5fc31d10 P cc38513f91d74c76a1260bde3b535c63: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:22.701089 8005 tablet_bootstrap.cc:654] T d4d4f250a3d845bd8d97f86c5fc31d10 P b80d695df1934011ad0ea7369d5010ef: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:22.701344 8004 log.cc:824] T d4d4f250a3d845bd8d97f86c5fc31d10 P cc38513f91d74c76a1260bde3b535c63: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:22.701386 8005 log.cc:824] T d4d4f250a3d845bd8d97f86c5fc31d10 P b80d695df1934011ad0ea7369d5010ef: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:22.702235 8005 tablet_bootstrap.cc:492] T d4d4f250a3d845bd8d97f86c5fc31d10 P b80d695df1934011ad0ea7369d5010ef: No bootstrap required, opened a new log
I20260619 17:02:22.702252 8004 tablet_bootstrap.cc:492] T d4d4f250a3d845bd8d97f86c5fc31d10 P cc38513f91d74c76a1260bde3b535c63: No bootstrap required, opened a new log
I20260619 17:02:22.702327 8004 ts_tablet_manager.cc:1397] T d4d4f250a3d845bd8d97f86c5fc31d10 P cc38513f91d74c76a1260bde3b535c63: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 17:02:22.702441 8005 ts_tablet_manager.cc:1397] T d4d4f250a3d845bd8d97f86c5fc31d10 P b80d695df1934011ad0ea7369d5010ef: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 17:02:22.704305 8008 tablet_bootstrap.cc:492] T d4d4f250a3d845bd8d97f86c5fc31d10 P 0fcb80ea14334ec19b5043e4c6b66bd1: Bootstrap starting.
I20260619 17:02:22.704883 8008 tablet_bootstrap.cc:654] T d4d4f250a3d845bd8d97f86c5fc31d10 P 0fcb80ea14334ec19b5043e4c6b66bd1: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:22.705196 8008 log.cc:824] T d4d4f250a3d845bd8d97f86c5fc31d10 P 0fcb80ea14334ec19b5043e4c6b66bd1: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:22.705229 8004 raft_consensus.cc:348] T d4d4f250a3d845bd8d97f86c5fc31d10 P cc38513f91d74c76a1260bde3b535c63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc38513f91d74c76a1260bde3b535c63" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34243 } } peers { permanent_uuid: "b80d695df1934011ad0ea7369d5010ef" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 38193 } } peers { permanent_uuid: "0fcb80ea14334ec19b5043e4c6b66bd1" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35335 } }
I20260619 17:02:22.705229 8005 raft_consensus.cc:348] T d4d4f250a3d845bd8d97f86c5fc31d10 P b80d695df1934011ad0ea7369d5010ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc38513f91d74c76a1260bde3b535c63" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34243 } } peers { permanent_uuid: "b80d695df1934011ad0ea7369d5010ef" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 38193 } } peers { permanent_uuid: "0fcb80ea14334ec19b5043e4c6b66bd1" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35335 } }
I20260619 17:02:22.705421 8005 raft_consensus.cc:374] T d4d4f250a3d845bd8d97f86c5fc31d10 P b80d695df1934011ad0ea7369d5010ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:22.705421 8004 raft_consensus.cc:374] T d4d4f250a3d845bd8d97f86c5fc31d10 P cc38513f91d74c76a1260bde3b535c63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:22.705469 8004 raft_consensus.cc:729] T d4d4f250a3d845bd8d97f86c5fc31d10 P cc38513f91d74c76a1260bde3b535c63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc38513f91d74c76a1260bde3b535c63, State: Initialized, Role: FOLLOWER
I20260619 17:02:22.705469 8005 raft_consensus.cc:729] T d4d4f250a3d845bd8d97f86c5fc31d10 P b80d695df1934011ad0ea7369d5010ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b80d695df1934011ad0ea7369d5010ef, State: Initialized, Role: FOLLOWER
I20260619 17:02:22.705595 8005 consensus_queue.cc:260] T d4d4f250a3d845bd8d97f86c5fc31d10 P b80d695df1934011ad0ea7369d5010ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc38513f91d74c76a1260bde3b535c63" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34243 } } peers { permanent_uuid: "b80d695df1934011ad0ea7369d5010ef" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 38193 } } peers { permanent_uuid: "0fcb80ea14334ec19b5043e4c6b66bd1" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35335 } }
I20260619 17:02:22.705615 8004 consensus_queue.cc:260] T d4d4f250a3d845bd8d97f86c5fc31d10 P cc38513f91d74c76a1260bde3b535c63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc38513f91d74c76a1260bde3b535c63" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34243 } } peers { permanent_uuid: "b80d695df1934011ad0ea7369d5010ef" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 38193 } } peers { permanent_uuid: "0fcb80ea14334ec19b5043e4c6b66bd1" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35335 } }
I20260619 17:02:22.705883 8004 ts_tablet_manager.cc:1428] T d4d4f250a3d845bd8d97f86c5fc31d10 P cc38513f91d74c76a1260bde3b535c63: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260619 17:02:22.705883 8005 ts_tablet_manager.cc:1428] T d4d4f250a3d845bd8d97f86c5fc31d10 P b80d695df1934011ad0ea7369d5010ef: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 17:02:22.706012 7859 heartbeater.cc:503] Master 127.20.231.190:45497 was elected leader, sending a full tablet report...
I20260619 17:02:22.706151 8008 tablet_bootstrap.cc:492] T d4d4f250a3d845bd8d97f86c5fc31d10 P 0fcb80ea14334ec19b5043e4c6b66bd1: No bootstrap required, opened a new log
I20260619 17:02:22.706239 8008 ts_tablet_manager.cc:1397] T d4d4f250a3d845bd8d97f86c5fc31d10 P 0fcb80ea14334ec19b5043e4c6b66bd1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 17:02:22.706944 7990 heartbeater.cc:503] Master 127.20.231.190:45497 was elected leader, sending a full tablet report...
I20260619 17:02:22.708508 8008 raft_consensus.cc:348] T d4d4f250a3d845bd8d97f86c5fc31d10 P 0fcb80ea14334ec19b5043e4c6b66bd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc38513f91d74c76a1260bde3b535c63" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34243 } } peers { permanent_uuid: "b80d695df1934011ad0ea7369d5010ef" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 38193 } } peers { permanent_uuid: "0fcb80ea14334ec19b5043e4c6b66bd1" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35335 } }
I20260619 17:02:22.708671 8008 raft_consensus.cc:374] T d4d4f250a3d845bd8d97f86c5fc31d10 P 0fcb80ea14334ec19b5043e4c6b66bd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:22.708715 8008 raft_consensus.cc:729] T d4d4f250a3d845bd8d97f86c5fc31d10 P 0fcb80ea14334ec19b5043e4c6b66bd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fcb80ea14334ec19b5043e4c6b66bd1, State: Initialized, Role: FOLLOWER
I20260619 17:02:22.708849 8008 consensus_queue.cc:260] T d4d4f250a3d845bd8d97f86c5fc31d10 P 0fcb80ea14334ec19b5043e4c6b66bd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc38513f91d74c76a1260bde3b535c63" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34243 } } peers { permanent_uuid: "b80d695df1934011ad0ea7369d5010ef" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 38193 } } peers { permanent_uuid: "0fcb80ea14334ec19b5043e4c6b66bd1" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35335 } }
I20260619 17:02:22.709146 8008 ts_tablet_manager.cc:1428] T d4d4f250a3d845bd8d97f86c5fc31d10 P 0fcb80ea14334ec19b5043e4c6b66bd1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 17:02:22.709215 7728 heartbeater.cc:503] Master 127.20.231.190:45497 was elected leader, sending a full tablet report...
W20260619 17:02:22.783044 7860 tablet.cc:2369] T d4d4f250a3d845bd8d97f86c5fc31d10 P cc38513f91d74c76a1260bde3b535c63: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 17:02:22.898993 7729 tablet.cc:2369] T d4d4f250a3d845bd8d97f86c5fc31d10 P 0fcb80ea14334ec19b5043e4c6b66bd1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 17:02:22.916214 7991 tablet.cc:2369] T d4d4f250a3d845bd8d97f86c5fc31d10 P b80d695df1934011ad0ea7369d5010ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 17:02:23.010668 8010 raft_consensus.cc:482] T d4d4f250a3d845bd8d97f86c5fc31d10 P b80d695df1934011ad0ea7369d5010ef [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 17:02:23.010815 8010 raft_consensus.cc:504] T d4d4f250a3d845bd8d97f86c5fc31d10 P b80d695df1934011ad0ea7369d5010ef [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc38513f91d74c76a1260bde3b535c63" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34243 } } peers { permanent_uuid: "b80d695df1934011ad0ea7369d5010ef" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 38193 } } peers { permanent_uuid: "0fcb80ea14334ec19b5043e4c6b66bd1" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35335 } }
I20260619 17:02:23.011193 8010 leader_election.cc:288] T d4d4f250a3d845bd8d97f86c5fc31d10 P b80d695df1934011ad0ea7369d5010ef [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc38513f91d74c76a1260bde3b535c63 (127.20.231.130:34243), 0fcb80ea14334ec19b5043e4c6b66bd1 (127.20.231.129:35335)
I20260619 17:02:23.015460 7992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:23.011451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:34243 (local address 127.20.231.131:56019)
0619 17:02:23.011520 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:23.011534 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:23.011601 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:23.011781 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:23.011782 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:23.011874 (+ 92us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:02:23.011948 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:23.011951 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:23.012912 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:23.012913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:23.014250 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:23.014254 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:23.015142 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:23.015143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:23.015228 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:23.015233 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 17:02:23.015303 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 17:02:23.015358 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":67}
I20260619 17:02:23.015481 8002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:23.011520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:56019 (local address 127.20.231.130:34243)
0619 17:02:23.011575 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:23.011578 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:23.011591 (+ 13us) server_negotiation.cc:407] Connection header received
0619 17:02:23.011670 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:23.011671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:23.011693 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:23.011746 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:02:23.012048 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:23.012808 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:23.014415 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:23.015064 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:23.015139 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:23.015146 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:23.015356 (+ 210us) server_negotiation.cc:299] Negotiation successful
0619 17:02:23.015405 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 17:02:23.015481 8001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:23.011652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:58679 (local address 127.20.231.129:35335)
0619 17:02:23.011707 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:23.011709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:23.011815 (+ 106us) server_negotiation.cc:407] Connection header received
0619 17:02:23.011855 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:23.011856 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:23.011866 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:23.011915 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:02:23.012057 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:23.012808 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:23.014410 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:23.015019 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:23.015113 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:23.015124 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:23.015362 (+ 238us) server_negotiation.cc:299] Negotiation successful
0619 17:02:23.015406 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 17:02:23.015466 8013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:23.011566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:35335 (local address 127.20.231.131:58679)
0619 17:02:23.011730 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:23.011741 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:23.011799 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:23.011919 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:23.011920 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:23.011922 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:02:23.011981 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:23.011982 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:23.012912 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:23.012913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:23.014250 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:23.014254 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:23.015138 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:23.015139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:23.015235 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:23.015238 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 17:02:23.015303 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 17:02:23.015355 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":53,"thread_start_us":108,"threads_started":1}
I20260619 17:02:23.015980 7682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d4f250a3d845bd8d97f86c5fc31d10" candidate_uuid: "b80d695df1934011ad0ea7369d5010ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fcb80ea14334ec19b5043e4c6b66bd1" is_pre_election: true
I20260619 17:02:23.015990 7812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d4f250a3d845bd8d97f86c5fc31d10" candidate_uuid: "b80d695df1934011ad0ea7369d5010ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc38513f91d74c76a1260bde3b535c63" is_pre_election: true
I20260619 17:02:23.016160 7682 raft_consensus.cc:2454] T d4d4f250a3d845bd8d97f86c5fc31d10 P 0fcb80ea14334ec19b5043e4c6b66bd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80d695df1934011ad0ea7369d5010ef in term 0.
I20260619 17:02:23.016188 7812 raft_consensus.cc:2454] T d4d4f250a3d845bd8d97f86c5fc31d10 P cc38513f91d74c76a1260bde3b535c63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b80d695df1934011ad0ea7369d5010ef in term 0.
I20260619 17:02:23.016423 7880 leader_election.cc:302] T d4d4f250a3d845bd8d97f86c5fc31d10 P b80d695df1934011ad0ea7369d5010ef [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fcb80ea14334ec19b5043e4c6b66bd1, b80d695df1934011ad0ea7369d5010ef; no voters:
I20260619 17:02:23.016595 8010 raft_consensus.cc:2781] T d4d4f250a3d845bd8d97f86c5fc31d10 P b80d695df1934011ad0ea7369d5010ef [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 17:02:23.016681 8010 raft_consensus.cc:482] T d4d4f250a3d845bd8d97f86c5fc31d10 P b80d695df1934011ad0ea7369d5010ef [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 17:02:23.016718 8010 raft_consensus.cc:3037] T d4d4f250a3d845bd8d97f86c5fc31d10 P b80d695df1934011ad0ea7369d5010ef [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:23.017484 8010 raft_consensus.cc:504] T d4d4f250a3d845bd8d97f86c5fc31d10 P b80d695df1934011ad0ea7369d5010ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc38513f91d74c76a1260bde3b535c63" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34243 } } peers { permanent_uuid: "b80d695df1934011ad0ea7369d5010ef" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 38193 } } peers { permanent_uuid: "0fcb80ea14334ec19b5043e4c6b66bd1" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35335 } }
I20260619 17:02:23.017642 8010 leader_election.cc:288] T d4d4f250a3d845bd8d97f86c5fc31d10 P b80d695df1934011ad0ea7369d5010ef [CANDIDATE]: Term 1 election: Requested vote from peers cc38513f91d74c76a1260bde3b535c63 (127.20.231.130:34243), 0fcb80ea14334ec19b5043e4c6b66bd1 (127.20.231.129:35335)
I20260619 17:02:23.017879 7682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d4f250a3d845bd8d97f86c5fc31d10" candidate_uuid: "b80d695df1934011ad0ea7369d5010ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fcb80ea14334ec19b5043e4c6b66bd1"
I20260619 17:02:23.017886 7812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d4d4f250a3d845bd8d97f86c5fc31d10" candidate_uuid: "b80d695df1934011ad0ea7369d5010ef" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc38513f91d74c76a1260bde3b535c63"
I20260619 17:02:23.017978 7682 raft_consensus.cc:3037] T d4d4f250a3d845bd8d97f86c5fc31d10 P 0fcb80ea14334ec19b5043e4c6b66bd1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:23.017975 7812 raft_consensus.cc:3037] T d4d4f250a3d845bd8d97f86c5fc31d10 P cc38513f91d74c76a1260bde3b535c63 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:23.018750 7682 raft_consensus.cc:2454] T d4d4f250a3d845bd8d97f86c5fc31d10 P 0fcb80ea14334ec19b5043e4c6b66bd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80d695df1934011ad0ea7369d5010ef in term 1.
I20260619 17:02:23.018752 7812 raft_consensus.cc:2454] T d4d4f250a3d845bd8d97f86c5fc31d10 P cc38513f91d74c76a1260bde3b535c63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b80d695df1934011ad0ea7369d5010ef in term 1.
I20260619 17:02:23.018947 7880 leader_election.cc:302] T d4d4f250a3d845bd8d97f86c5fc31d10 P b80d695df1934011ad0ea7369d5010ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b80d695df1934011ad0ea7369d5010ef, cc38513f91d74c76a1260bde3b535c63; no voters:
I20260619 17:02:23.019061 8010 raft_consensus.cc:2781] T d4d4f250a3d845bd8d97f86c5fc31d10 P b80d695df1934011ad0ea7369d5010ef [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:02:23.019240 8010 raft_consensus.cc:686] T d4d4f250a3d845bd8d97f86c5fc31d10 P b80d695df1934011ad0ea7369d5010ef [term 1 LEADER]: Becoming Leader. State: Replica: b80d695df1934011ad0ea7369d5010ef, State: Running, Role: LEADER
I20260619 17:02:23.019410 8010 consensus_queue.cc:237] T d4d4f250a3d845bd8d97f86c5fc31d10 P b80d695df1934011ad0ea7369d5010ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc38513f91d74c76a1260bde3b535c63" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34243 } } peers { permanent_uuid: "b80d695df1934011ad0ea7369d5010ef" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 38193 } } peers { permanent_uuid: "0fcb80ea14334ec19b5043e4c6b66bd1" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35335 } }
I20260619 17:02:23.020291 7539 catalog_manager.cc:5495] T d4d4f250a3d845bd8d97f86c5fc31d10 P b80d695df1934011ad0ea7369d5010ef reported cstate change: term changed from 0 to 1, leader changed from <none> to b80d695df1934011ad0ea7369d5010ef (127.20.231.131). New cstate: current_term: 1 leader_uuid: "b80d695df1934011ad0ea7369d5010ef" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc38513f91d74c76a1260bde3b535c63" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34243 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b80d695df1934011ad0ea7369d5010ef" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 38193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0fcb80ea14334ec19b5043e4c6b66bd1" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35335 } health_report { overall_health: UNKNOWN } } }
I20260619 17:02:23.055460 8003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:23.052715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43660 (local address 127.20.231.131:38193)
0619 17:02:23.052762 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:23.052764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:23.052869 (+ 105us) server_negotiation.cc:407] Connection header received
0619 17:02:23.052942 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:23.052943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:23.052963 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:23.053025 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 17:02:23.053160 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:23.053619 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:23.054505 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:23.054697 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:23.054759 (+ 62us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:23.055038 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 17:02:23.055128 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 17:02:23.055207 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:23.055311 (+ 104us) server_negotiation.cc:299] Negotiation successful
0619 17:02:23.055319 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 17:02:23.057651 7682 raft_consensus.cc:1261] T d4d4f250a3d845bd8d97f86c5fc31d10 P 0fcb80ea14334ec19b5043e4c6b66bd1 [term 1 FOLLOWER]: Refusing update from remote peer b80d695df1934011ad0ea7369d5010ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 17:02:23.057685 7812 raft_consensus.cc:1261] T d4d4f250a3d845bd8d97f86c5fc31d10 P cc38513f91d74c76a1260bde3b535c63 [term 1 FOLLOWER]: Refusing update from remote peer b80d695df1934011ad0ea7369d5010ef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 17:02:23.058035 8014 consensus_queue.cc:1035] T d4d4f250a3d845bd8d97f86c5fc31d10 P b80d695df1934011ad0ea7369d5010ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc38513f91d74c76a1260bde3b535c63" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 34243 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 17:02:23.058187 8010 consensus_queue.cc:1035] T d4d4f250a3d845bd8d97f86c5fc31d10 P b80d695df1934011ad0ea7369d5010ef [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fcb80ea14334ec19b5043e4c6b66bd1" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 35335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 17:02:23.060925 8017 mvcc.cc:205] Tried to move back new op lower bound from 7298615472360448000 to 7298615472208105472. Current Snapshot: MvccSnapshot[applied={T|T < 7298615472360448000}]
I20260619 17:02:23.062460 8020 mvcc.cc:205] Tried to move back new op lower bound from 7298615472360448000 to 7298615472208105472. Current Snapshot: MvccSnapshot[applied={T|T < 7298615472360448000}]
I20260619 17:02:23.065083 7540 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 17:02:23.065193 7540 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46548:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 17:02:23.066071 7540 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0d15436f445f4ce390476382a8eafe3d: Sending DeleteTablet for 3 replicas of tablet d4d4f250a3d845bd8d97f86c5fc31d10
I20260619 17:02:23.066437 7924 tablet_service.cc:1514] Processing DeleteTablet for tablet d4d4f250a3d845bd8d97f86c5fc31d10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:02:23 UTC) from {username='slave'} at 127.0.0.1:43646
I20260619 17:02:23.066485 7662 tablet_service.cc:1514] Processing DeleteTablet for tablet d4d4f250a3d845bd8d97f86c5fc31d10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:02:23 UTC) from {username='slave'} at 127.0.0.1:37054
I20260619 17:02:23.066534 7793 tablet_service.cc:1514] Processing DeleteTablet for tablet d4d4f250a3d845bd8d97f86c5fc31d10 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:02:23 UTC) from {username='slave'} at 127.0.0.1:42036
I20260619 17:02:23.066825 8026 tablet_replica.cc:330] T d4d4f250a3d845bd8d97f86c5fc31d10 P b80d695df1934011ad0ea7369d5010ef: stopping tablet replica
I20260619 17:02:23.067013 8026 raft_consensus.cc:2229] T d4d4f250a3d845bd8d97f86c5fc31d10 P b80d695df1934011ad0ea7369d5010ef [term 1 LEADER]: Raft consensus shutting down.
I20260619 17:02:23.067130 8026 raft_consensus.cc:2258] T d4d4f250a3d845bd8d97f86c5fc31d10 P b80d695df1934011ad0ea7369d5010ef [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 17:02:23.067762 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 7600
I20260619 17:02:23.068364 8026 ts_tablet_manager.cc:1905] T d4d4f250a3d845bd8d97f86c5fc31d10 P b80d695df1934011ad0ea7369d5010ef: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 17:02:23.070536 8026 ts_tablet_manager.cc:1918] T d4d4f250a3d845bd8d97f86c5fc31d10 P b80d695df1934011ad0ea7369d5010ef: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 17:02:23.070621 8026 log.cc:1196] T d4d4f250a3d845bd8d97f86c5fc31d10 P b80d695df1934011ad0ea7369d5010ef: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781888301345144-21406-0/minicluster-data/ts-2/wal/wals/d4d4f250a3d845bd8d97f86c5fc31d10
I20260619 17:02:23.070911 8026 ts_tablet_manager.cc:1939] T d4d4f250a3d845bd8d97f86c5fc31d10 P b80d695df1934011ad0ea7369d5010ef: Deleting consensus metadata
I20260619 17:02:23.071398 7525 catalog_manager.cc:4841] TS b80d695df1934011ad0ea7369d5010ef (127.20.231.131:38193): tablet d4d4f250a3d845bd8d97f86c5fc31d10 (table test-table [id=b80b68753b1340edb6173f8815bf7733]) successfully deleted
I20260619 17:02:23.075049 8028 tablet_replica.cc:330] T d4d4f250a3d845bd8d97f86c5fc31d10 P cc38513f91d74c76a1260bde3b535c63: stopping tablet replica
I20260619 17:02:23.075206 8028 raft_consensus.cc:2229] T d4d4f250a3d845bd8d97f86c5fc31d10 P cc38513f91d74c76a1260bde3b535c63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 17:02:23.075348 8028 raft_consensus.cc:2258] T d4d4f250a3d845bd8d97f86c5fc31d10 P cc38513f91d74c76a1260bde3b535c63 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260619 17:02:23.075559 7527 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.231.129:35335 (error 108)
I20260619 17:02:23.075897 8028 ts_tablet_manager.cc:1905] T d4d4f250a3d845bd8d97f86c5fc31d10 P cc38513f91d74c76a1260bde3b535c63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 17:02:23.076109 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 7731
I20260619 17:02:23.076184 7998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:23.076023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:35335 (local address 127.0.0.1:37060)
0619 17:02:23.076107 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:23.076150 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.129:35335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260619 17:02:23.076726 7527 catalog_manager.cc:4584] TS 0fcb80ea14334ec19b5043e4c6b66bd1 (127.20.231.129:35335): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d4d4f250a3d845bd8d97f86c5fc31d10: Network error: Client connection negotiation failed: client connection to 127.20.231.129:35335: connect: Connection refused (error 111)
I20260619 17:02:23.082475 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 7862
I20260619 17:02:23.083266 7998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:23.082535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:34243 (local address 127.0.0.1:42042)
0619 17:02:23.083185 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:23.083219 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.130:34243: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":624}
I20260619 17:02:23.087761 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 7509
2026-06-19T17:02:23Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1012 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 17:02:23 dist-test-slave-vnbj krb5kdc[8032](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 17:02:23 dist-test-slave-vnbj krb5kdc[8032](info): set up 2 sockets
Jun 19 17:02:23 dist-test-slave-vnbj krb5kdc[8032](info): commencing operation
krb5kdc: starting...
W20260619 17:02:24.171008 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 17:02:24 dist-test-slave-vnbj krb5kdc[8032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888544, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T17:02:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T17:02:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.231.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
I20260619 17:02:24.307510 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:39035
--webserver_interface=127.20.231.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:41659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:39035
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:02:24.393795 8045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:24.394071 8045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:24.394112 8045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:24.395464 8045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:02:24.395551 8045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:24.395576 8045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:24.395597 8045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:02:24.395629 8045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:02:24.397230 8045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:41659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:39035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:39035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:24.397532 8045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:24.397795 8045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:24.403276 8050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:24.403237 8051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:24.403237 8053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:24.403568 8045 server_base.cc:1034] running on GCE node
I20260619 17:02:24.403857 8045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:24.404237 8045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:24.405417 8045 hybrid_clock.cc:648] HybridClock initialized: now 1781888544405392 us; error 44 us; skew 500 ppm
Jun 19 17:02:24 dist-test-slave-vnbj krb5kdc[8032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888544, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:24.408509 8045 init.cc:377] Logged in from keytab as kudu/127.20.231.190@KRBTEST.COM (short username kudu)
I20260619 17:02:24.409068 8045 webserver.cc:460] Webserver started at http://127.20.231.190:41535/ using document root <none> and password file <none>
I20260619 17:02:24.409313 8045 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:24.409363 8045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:24.409477 8045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:24.410646 8045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "2e56a3932c054215afa01ef7ecf386e1"
format_stamp: "Formatted at 2026-06-19 17:02:24 on dist-test-slave-vnbj"
server_key: "1d1be361ba5b7d8881e00fd7d09f45e3"
server_key_iv: "8f17921685ed93d4fee1168aa1424f70"
server_key_version: "encryptionkey@0"
I20260619 17:02:24.411160 8045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "2e56a3932c054215afa01ef7ecf386e1"
format_stamp: "Formatted at 2026-06-19 17:02:24 on dist-test-slave-vnbj"
server_key: "1d1be361ba5b7d8881e00fd7d09f45e3"
server_key_iv: "8f17921685ed93d4fee1168aa1424f70"
server_key_version: "encryptionkey@0"
I20260619 17:02:24.413110 8045 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260619 17:02:24.414479 8060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:24.415131 8045 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:24.415295 8045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "2e56a3932c054215afa01ef7ecf386e1"
format_stamp: "Formatted at 2026-06-19 17:02:24 on dist-test-slave-vnbj"
server_key: "1d1be361ba5b7d8881e00fd7d09f45e3"
server_key_iv: "8f17921685ed93d4fee1168aa1424f70"
server_key_version: "encryptionkey@0"
I20260619 17:02:24.415470 8045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:24.427665 8045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:24.430414 8045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:24.430596 8045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:24.435631 8045 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:39035
I20260619 17:02:24.435710 8112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:39035 every 8 connection(s)
I20260619 17:02:24.436136 8045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 17:02:24.437192 8113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:24.440285 8113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e56a3932c054215afa01ef7ecf386e1: Bootstrap starting.
I20260619 17:02:24.441062 8113 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e56a3932c054215afa01ef7ecf386e1: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:24.441393 8113 log.cc:824] T 00000000000000000000000000000000 P 2e56a3932c054215afa01ef7ecf386e1: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:24.442282 8113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e56a3932c054215afa01ef7ecf386e1: No bootstrap required, opened a new log
I20260619 17:02:24.444036 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 8045
I20260619 17:02:24.444144 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 17:02:24.444433 21406 external_mini_cluster.cc:1428] Setting key 3731c94b907157a2abca25fdfab56fc9
I20260619 17:02:24.444455 8113 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e56a3932c054215afa01ef7ecf386e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e56a3932c054215afa01ef7ecf386e1" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 39035 } }
I20260619 17:02:24.444696 8113 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e56a3932c054215afa01ef7ecf386e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:24.444753 8113 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e56a3932c054215afa01ef7ecf386e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e56a3932c054215afa01ef7ecf386e1, State: Initialized, Role: FOLLOWER
I20260619 17:02:24.444918 8113 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e56a3932c054215afa01ef7ecf386e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e56a3932c054215afa01ef7ecf386e1" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 39035 } }
I20260619 17:02:24.445040 8113 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e56a3932c054215afa01ef7ecf386e1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 17:02:24.445086 8113 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e56a3932c054215afa01ef7ecf386e1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 17:02:24.445147 8113 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e56a3932c054215afa01ef7ecf386e1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:24.446527 8113 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e56a3932c054215afa01ef7ecf386e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e56a3932c054215afa01ef7ecf386e1" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 39035 } }
I20260619 17:02:24.446671 8113 leader_election.cc:302] T 00000000000000000000000000000000 P 2e56a3932c054215afa01ef7ecf386e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e56a3932c054215afa01ef7ecf386e1; no voters:
I20260619 17:02:24.446949 8113 leader_election.cc:288] T 00000000000000000000000000000000 P 2e56a3932c054215afa01ef7ecf386e1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 17:02:24.447001 8118 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e56a3932c054215afa01ef7ecf386e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:02:24.447173 8118 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e56a3932c054215afa01ef7ecf386e1 [term 1 LEADER]: Becoming Leader. State: Replica: 2e56a3932c054215afa01ef7ecf386e1, State: Running, Role: LEADER
I20260619 17:02:24.447269 8113 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e56a3932c054215afa01ef7ecf386e1 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:02:24.447296 8118 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e56a3932c054215afa01ef7ecf386e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e56a3932c054215afa01ef7ecf386e1" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 39035 } }
I20260619 17:02:24.447640 8118 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e56a3932c054215afa01ef7ecf386e1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e56a3932c054215afa01ef7ecf386e1. Latest consensus state: current_term: 1 leader_uuid: "2e56a3932c054215afa01ef7ecf386e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e56a3932c054215afa01ef7ecf386e1" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 39035 } } }
I20260619 17:02:24.447721 8118 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e56a3932c054215afa01ef7ecf386e1 [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:24.447649 8120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e56a3932c054215afa01ef7ecf386e1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e56a3932c054215afa01ef7ecf386e1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e56a3932c054215afa01ef7ecf386e1" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 39035 } } }
I20260619 17:02:24.447950 8120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e56a3932c054215afa01ef7ecf386e1 [sys.catalog]: This master's current role is: LEADER
W20260619 17:02:24.449450 8134 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2e56a3932c054215afa01ef7ecf386e1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 17:02:24.449528 8134 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 17:02:24.449579 8124 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 17:02:24.449750 8124 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 17:02:24.451589 8124 catalog_manager.cc:1347] Generated new cluster ID: 66f62ab241dc41ee89a522a3dd96ab30
I20260619 17:02:24.451649 8124 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 17:02:24 dist-test-slave-vnbj krb5kdc[8032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888544, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:02:24.456992 8117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:24.445076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48200 (local address 127.20.231.190:39035)
0619 17:02:24.445327 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:24.445336 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:24.445365 (+ 29us) server_negotiation.cc:407] Connection header received
0619 17:02:24.445491 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:24.445497 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:24.445553 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:24.445689 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:24.446732 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:24.448356 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:24.449586 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:24.450161 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:24.450294 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:24.453231 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:24.453236 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:24.453237 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:24.453251 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:24.455231 (+ 1980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:24.455679 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:24.455682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:24.455683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:24.455796 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:24.456037 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:24.456038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:24.456038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:24.456307 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:24.456515 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:24.456642 (+ 127us) server_negotiation.cc:299] Negotiation successful
0619 17:02:24.456804 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":130,"threads_started":1}
I20260619 17:02:24.472682 8124 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 17:02:24.473372 8124 catalog_manager.cc:1504] Loading token signing keys...
I20260619 17:02:24.477802 8124 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e56a3932c054215afa01ef7ecf386e1: Generated new TSK 0
I20260619 17:02:24.478134 8124 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.20.231.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab.
Entry for principal kudu/127.20.231.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab.
I20260619 17:02:24.511071 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.129:0
--local_ip_for_outbound_sockets=127.20.231.129
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:39035
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:41659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:02:24.615415 8139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:24.615634 8139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:24.615661 8139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:24.616912 8139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:24.616962 8139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:24.616990 8139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.129
I20260619 17:02:24.618537 8139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:41659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.129.keytab
--principal=kudu/127.20.231.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:39035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.129
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:24.618818 8139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:24.619112 8139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:24.624454 8144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:24.624444 8147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:24.624444 8145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:24.624944 8139 server_base.cc:1034] running on GCE node
I20260619 17:02:24.625165 8139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:24.625442 8139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:24.626633 8139 hybrid_clock.cc:648] HybridClock initialized: now 1781888544626611 us; error 48 us; skew 500 ppm
Jun 19 17:02:24 dist-test-slave-vnbj krb5kdc[8032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888544, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:24.629487 8139 init.cc:377] Logged in from keytab as kudu/127.20.231.129@KRBTEST.COM (short username kudu)
I20260619 17:02:24.629901 8139 webserver.cc:460] Webserver started at http://127.20.231.129:41669/ using document root <none> and password file <none>
I20260619 17:02:24.630108 8139 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:24.630149 8139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:24.630239 8139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:24.631417 8139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-0/data/instance:
uuid: "c4c55ccad53545b7a1cc12d429468642"
format_stamp: "Formatted at 2026-06-19 17:02:24 on dist-test-slave-vnbj"
server_key: "321ba482b7d2a0f7af7519b8d662a52b"
server_key_iv: "6ef33b3b4fcf09f62227691861715682"
server_key_version: "encryptionkey@0"
I20260619 17:02:24.631731 8139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance:
uuid: "c4c55ccad53545b7a1cc12d429468642"
format_stamp: "Formatted at 2026-06-19 17:02:24 on dist-test-slave-vnbj"
server_key: "321ba482b7d2a0f7af7519b8d662a52b"
server_key_iv: "6ef33b3b4fcf09f62227691861715682"
server_key_version: "encryptionkey@0"
I20260619 17:02:24.633176 8139 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 17:02:24.633968 8154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:24.634197 8139 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:24.634299 8139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-0/wal
uuid: "c4c55ccad53545b7a1cc12d429468642"
format_stamp: "Formatted at 2026-06-19 17:02:24 on dist-test-slave-vnbj"
server_key: "321ba482b7d2a0f7af7519b8d662a52b"
server_key_iv: "6ef33b3b4fcf09f62227691861715682"
server_key_version: "encryptionkey@0"
I20260619 17:02:24.634362 8139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:24.654692 8139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:24.657081 8139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:24.657208 8139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:24.657411 8139 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:02:24.657768 8139 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:02:24.657807 8139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:24.657832 8139 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:02:24.657887 8139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:24.664294 8139 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.129:37587
I20260619 17:02:24.664402 8267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.129:37587 every 8 connection(s)
I20260619 17:02:24.664737 8139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
I20260619 17:02:24.667757 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 8139
I20260619 17:02:24.667876 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance
I20260619 17:02:24.668069 21406 external_mini_cluster.cc:1428] Setting key 18318ea89df88add855f3392fc488f01
Jun 19 17:02:24 dist-test-slave-vnbj krb5kdc[8032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888544, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.129@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:02:24.676487 8117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:24.665531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:44981 (local address 127.20.231.190:39035)
0619 17:02:24.665597 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:24.665600 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:24.665857 (+ 257us) server_negotiation.cc:407] Connection header received
0619 17:02:24.666388 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:24.666389 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:24.666407 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:24.666460 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:24.667501 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:24.668335 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:24.669715 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:24.670061 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:24.670139 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:24.673127 (+ 2988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:24.673129 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:24.673130 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:24.673138 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:24.674895 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:24.675381 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:24.675383 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:24.675383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:24.675448 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:24.675713 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:24.675714 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:24.675715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:24.675910 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:24.675995 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:24.676302 (+ 307us) server_negotiation.cc:299] Negotiation successful
0619 17:02:24.676389 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 17:02:24.676720 8270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:24.665502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:39035 (local address 127.20.231.129:44981)
0619 17:02:24.665650 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:24.665696 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:24.666230 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:24.666545 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:24.666554 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:24.667182 (+ 628us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:24.667394 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:24.667398 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:24.668447 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:24.668448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:24.669606 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:24.669609 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:24.670179 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:24.670180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:24.670371 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:24.671043 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:02:24.671055 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:24.672983 (+ 1928us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:24.675007 (+ 2024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:24.675016 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:24.675018 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:24.675278 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:24.675519 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:24.675520 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:24.675520 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:24.675643 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:24.675986 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:24.675994 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:24.676148 (+ 154us) client_negotiation.cc:769] Sending connection context
0619 17:02:24.676279 (+ 131us) client_negotiation.cc:240] Negotiation successful
0619 17:02:24.676455 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":68,"threads_started":1}
WARNING: no policy specified for kudu/127.20.231.130@KRBTEST.COM; defaulting to no policy
I20260619 17:02:24.677254 8268 heartbeater.cc:347] Connected to a master server at 127.20.231.190:39035
I20260619 17:02:24.677377 8268 heartbeater.cc:464] Registering TS with master...
I20260619 17:02:24.677558 8268 heartbeater.cc:511] Master 127.20.231.190:39035 requested a full tablet report, sending...
I20260619 17:02:24.678038 8077 ts_manager.cc:194] Registered new tserver with Master: c4c55ccad53545b7a1cc12d429468642 (127.20.231.129:37587)
I20260619 17:02:24.678782 8077 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.129@KRBTEST.COM'} at 127.20.231.129:44981
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab.
Entry for principal kudu/127.20.231.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab.
I20260619 17:02:24.699717 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.130:0
--local_ip_for_outbound_sockets=127.20.231.130
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:39035
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:41659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:02:24.782806 8273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:24.783095 8273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:24.783138 8273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:24.784368 8273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:24.784448 8273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:24.784495 8273 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.130
I20260619 17:02:24.786051 8273 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:41659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.130.keytab
--principal=kudu/127.20.231.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:39035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.130
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:24.786357 8273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:24.786612 8273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:24.791977 8279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:24.792098 8281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:24.792023 8278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:24.792204 8273 server_base.cc:1034] running on GCE node
I20260619 17:02:24.792430 8273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:24.792702 8273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:24.793901 8273 hybrid_clock.cc:648] HybridClock initialized: now 1781888544793877 us; error 46 us; skew 500 ppm
Jun 19 17:02:24 dist-test-slave-vnbj krb5kdc[8032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888544, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:24.796729 8273 init.cc:377] Logged in from keytab as kudu/127.20.231.130@KRBTEST.COM (short username kudu)
I20260619 17:02:24.797247 8273 webserver.cc:460] Webserver started at http://127.20.231.130:36749/ using document root <none> and password file <none>
I20260619 17:02:24.797518 8273 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:24.797586 8273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:24.797745 8273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:24.798971 8273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-1/data/instance:
uuid: "4ad9430654f243c3968b4e6607266dc2"
format_stamp: "Formatted at 2026-06-19 17:02:24 on dist-test-slave-vnbj"
server_key: "c03144efb4df47889672a1fa26f0b2be"
server_key_iv: "51be55f74f04dbf108800f05b1325307"
server_key_version: "encryptionkey@0"
I20260619 17:02:24.799324 8273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance:
uuid: "4ad9430654f243c3968b4e6607266dc2"
format_stamp: "Formatted at 2026-06-19 17:02:24 on dist-test-slave-vnbj"
server_key: "c03144efb4df47889672a1fa26f0b2be"
server_key_iv: "51be55f74f04dbf108800f05b1325307"
server_key_version: "encryptionkey@0"
I20260619 17:02:24.800680 8273 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 17:02:24.801630 8288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:24.801894 8273 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:24.802016 8273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-1/wal
uuid: "4ad9430654f243c3968b4e6607266dc2"
format_stamp: "Formatted at 2026-06-19 17:02:24 on dist-test-slave-vnbj"
server_key: "c03144efb4df47889672a1fa26f0b2be"
server_key_iv: "51be55f74f04dbf108800f05b1325307"
server_key_version: "encryptionkey@0"
I20260619 17:02:24.802095 8273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:24.816725 8273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:24.819126 8273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:24.819311 8273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:24.819582 8273 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:02:24.819938 8273 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:02:24.819994 8273 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:24.820036 8273 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:02:24.820066 8273 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:24.826658 8273 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.130:32969
I20260619 17:02:24.826745 8401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.130:32969 every 8 connection(s)
I20260619 17:02:24.827186 8273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
Jun 19 17:02:24 dist-test-slave-vnbj krb5kdc[8032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888544, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.130@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:02:24.834916 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 8273
I20260619 17:02:24.835014 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance
I20260619 17:02:24.835220 21406 external_mini_cluster.cc:1428] Setting key ea1b6ec59ef56da2bc588bd00cda9894
I20260619 17:02:24.838538 8117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:24.828412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:33397 (local address 127.20.231.190:39035)
0619 17:02:24.828465 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:24.828467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:24.828479 (+ 12us) server_negotiation.cc:407] Connection header received
0619 17:02:24.829124 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:24.829126 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:24.829138 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:24.829215 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:24.830526 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:24.831115 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:24.832210 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:24.832416 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:24.832486 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:24.834698 (+ 2212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:24.834699 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:24.834699 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:24.834707 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:24.836679 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:24.837277 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:24.837278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:24.837278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:24.837355 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:24.837741 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:24.837742 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:24.837743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:24.837925 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:24.838000 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:24.838395 (+ 395us) server_negotiation.cc:299] Negotiation successful
0619 17:02:24.838470 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260619 17:02:24.838735 8404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:24.828056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:39035 (local address 127.20.231.130:33397)
0619 17:02:24.828221 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:24.828278 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:24.828977 (+ 699us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:24.829364 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:24.829368 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:24.830149 (+ 781us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:24.830418 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:24.830424 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:24.831201 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:24.831202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:24.832100 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:24.832103 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:24.832502 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:24.832502 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:24.832611 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:24.833179 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:02:24.833188 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:24.834591 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:24.836825 (+ 2234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:24.836834 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:24.836837 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:24.837183 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:24.837432 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:24.837433 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:24.837433 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:24.837631 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:24.838006 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:24.838012 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:24.838207 (+ 195us) client_negotiation.cc:769] Sending connection context
0619 17:02:24.838344 (+ 137us) client_negotiation.cc:240] Negotiation successful
0619 17:02:24.838488 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":81,"threads_started":1}
I20260619 17:02:24.839345 8402 heartbeater.cc:347] Connected to a master server at 127.20.231.190:39035
I20260619 17:02:24.839448 8402 heartbeater.cc:464] Registering TS with master...
I20260619 17:02:24.839654 8402 heartbeater.cc:511] Master 127.20.231.190:39035 requested a full tablet report, sending...
I20260619 17:02:24.840142 8077 ts_manager.cc:194] Registered new tserver with Master: 4ad9430654f243c3968b4e6607266dc2 (127.20.231.130:32969)
I20260619 17:02:24.840574 8077 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.130@KRBTEST.COM'} at 127.20.231.130:33397
WARNING: no policy specified for kudu/127.20.231.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab.
Entry for principal kudu/127.20.231.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab.
I20260619 17:02:24.866503 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.131:0
--local_ip_for_outbound_sockets=127.20.231.131
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:39035
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:41659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:02:24.951956 8407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:24.952209 8407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:24.952252 8407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:24.953603 8407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:24.953781 8407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:24.953877 8407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.131
I20260619 17:02:24.955835 8407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:41659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.131.keytab
--principal=kudu/127.20.231.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:39035
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.131
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:24.956140 8407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:24.956403 8407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:24.962013 8412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:24.961989 8415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:24.962019 8407 server_base.cc:1034] running on GCE node
W20260619 17:02:24.961985 8413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:24.962437 8407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:24.962651 8407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:24.963903 8407 hybrid_clock.cc:648] HybridClock initialized: now 1781888544963876 us; error 43 us; skew 500 ppm
Jun 19 17:02:24 dist-test-slave-vnbj krb5kdc[8032](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888544, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:24.966694 8407 init.cc:377] Logged in from keytab as kudu/127.20.231.131@KRBTEST.COM (short username kudu)
I20260619 17:02:24.967193 8407 webserver.cc:460] Webserver started at http://127.20.231.131:39661/ using document root <none> and password file <none>
I20260619 17:02:24.967453 8407 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:24.967504 8407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:24.967636 8407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:24.968711 8407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-2/data/instance:
uuid: "429ac84ef650434f9725b23b3424c90f"
format_stamp: "Formatted at 2026-06-19 17:02:24 on dist-test-slave-vnbj"
server_key: "9d36c5e705c00e7c27d36eaee1aa4a12"
server_key_iv: "8f36b3a7bbb6c4846494b84cb5d7af7c"
server_key_version: "encryptionkey@0"
I20260619 17:02:24.969079 8407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance:
uuid: "429ac84ef650434f9725b23b3424c90f"
format_stamp: "Formatted at 2026-06-19 17:02:24 on dist-test-slave-vnbj"
server_key: "9d36c5e705c00e7c27d36eaee1aa4a12"
server_key_iv: "8f36b3a7bbb6c4846494b84cb5d7af7c"
server_key_version: "encryptionkey@0"
I20260619 17:02:24.970496 8407 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 17:02:24.971447 8422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:24.971676 8407 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 17:02:24.971755 8407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-2/wal
uuid: "429ac84ef650434f9725b23b3424c90f"
format_stamp: "Formatted at 2026-06-19 17:02:24 on dist-test-slave-vnbj"
server_key: "9d36c5e705c00e7c27d36eaee1aa4a12"
server_key_iv: "8f36b3a7bbb6c4846494b84cb5d7af7c"
server_key_version: "encryptionkey@0"
I20260619 17:02:24.971851 8407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:24.988843 8407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:24.991003 8407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:24.991163 8407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:24.991448 8407 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:02:24.991801 8407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:02:24.991839 8407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:24.991863 8407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:02:24.991875 8407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:24.998021 8407 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.131:40275
I20260619 17:02:24.998083 8535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.131:40275 every 8 connection(s)
I20260619 17:02:24.998454 8407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
I20260619 17:02:25.001992 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 8407
I20260619 17:02:25.002080 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance
I20260619 17:02:25.002278 21406 external_mini_cluster.cc:1428] Setting key b71cefcd2fea24560df94484cb806038
Jun 19 17:02:25 dist-test-slave-vnbj krb5kdc[8032](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888544, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.131@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:02:25.008996 8117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:24.999256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:60231 (local address 127.20.231.190:39035)
0619 17:02:24.999299 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:24.999301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:24.999558 (+ 257us) server_negotiation.cc:407] Connection header received
0619 17:02:25.000064 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:25.000065 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:25.000075 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:25.000134 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:25.001128 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.001656 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.003136 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.003362 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.003414 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:25.005735 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:25.005736 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:25.005737 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:25.005744 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:25.007368 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:25.007845 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:25.007845 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:25.007846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:25.007897 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:25.008209 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:25.008210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:25.008210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:25.008378 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:25.008445 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:25.008821 (+ 376us) server_negotiation.cc:299] Negotiation successful
0619 17:02:25.008918 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260619 17:02:25.009166 8538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:24.999215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:39035 (local address 127.20.231.131:60231)
0619 17:02:24.999423 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:24.999468 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:24.999970 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:25.000197 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:25.000200 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:25.000811 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:25.001018 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:25.001022 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.001896 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.001898 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:25.003024 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:25.003028 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.003437 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.003439 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:25.003554 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:25.004139 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:02:25.004147 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:25.005627 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:25.007511 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:25.007517 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:25.007519 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:25.007758 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:25.007977 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:25.007978 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:25.007979 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:25.008110 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:25.008475 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:25.008482 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:25.008619 (+ 137us) client_negotiation.cc:769] Sending connection context
0619 17:02:25.008763 (+ 144us) client_negotiation.cc:240] Negotiation successful
0619 17:02:25.008921 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":70,"threads_started":1}
I20260619 17:02:25.009845 8536 heartbeater.cc:347] Connected to a master server at 127.20.231.190:39035
I20260619 17:02:25.009999 8536 heartbeater.cc:464] Registering TS with master...
I20260619 17:02:25.010156 8536 heartbeater.cc:511] Master 127.20.231.190:39035 requested a full tablet report, sending...
I20260619 17:02:25.010609 8077 ts_manager.cc:194] Registered new tserver with Master: 429ac84ef650434f9725b23b3424c90f (127.20.231.131:40275)
I20260619 17:02:25.011195 8077 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.20.231.131@KRBTEST.COM'} at 127.20.231.131:60231
I20260619 17:02:25.014163 21406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 17:02:25.023172 8117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:25.015720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48212 (local address 127.20.231.190:39035)
0619 17:02:25.015787 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:25.015789 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:25.015899 (+ 110us) server_negotiation.cc:407] Connection header received
0619 17:02:25.015960 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:25.015961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:25.015971 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:25.016022 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:25.016590 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.017108 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.017999 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.018216 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.018288 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:25.019889 (+ 1601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:25.019891 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:25.019891 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:25.019899 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:25.021667 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:25.022060 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:25.022062 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:25.022062 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:25.022131 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:25.022383 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:25.022385 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:25.022385 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:25.022560 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:25.022680 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:25.022965 (+ 285us) server_negotiation.cc:299] Negotiation successful
0619 17:02:25.023055 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 17:02:25.024995 8077 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48212:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 17:02:25.026749 8077 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 17:02:25.033370 8548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:25.029273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:32969 (local address 127.0.0.1:41608)
0619 17:02:25.029445 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:25.029509 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:25.029590 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:25.029867 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:25.029871 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:25.029873 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:02:25.029997 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:25.029999 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.031196 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.031197 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:25.032197 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:25.032200 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.032987 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.032989 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:25.033079 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:25.033094 (+ 15us) client_negotiation.cc:769] Sending connection context
0619 17:02:25.033207 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 17:02:25.033268 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":65,"threads_started":1}
I20260619 17:02:25.033365 8551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:25.029485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41608 (local address 127.20.231.130:32969)
0619 17:02:25.029686 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:25.029692 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:25.029703 (+ 11us) server_negotiation.cc:407] Connection header received
0619 17:02:25.029742 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:25.029747 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:25.029770 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:25.029883 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:02:25.030093 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.031080 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.032267 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.032885 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.032991 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:25.033055 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:25.033227 (+ 172us) server_negotiation.cc:299] Negotiation successful
0619 17:02:25.033269 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":83,"threads_started":1}
I20260619 17:02:25.034157 8336 tablet_service.cc:1467] Processing CreateTablet for tablet 06eba101256f436a8a83a13983faf2b2 (DEFAULT_TABLE table=test-table [id=ce4c24e1264f4672bf0e10fa2b564214]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 17:02:25.034461 8549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:25.029405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43294 (local address 127.20.231.131:40275)
0619 17:02:25.029772 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:25.029778 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:25.030143 (+ 365us) server_negotiation.cc:407] Connection header received
0619 17:02:25.030189 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:25.030191 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:25.030206 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:25.030287 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:02:25.030508 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.031647 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.033106 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.033987 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.034064 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:25.034116 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:25.034264 (+ 148us) server_negotiation.cc:299] Negotiation successful
0619 17:02:25.034318 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":109,"threads_started":1}
I20260619 17:02:25.034603 8552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:25.029742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51238 (local address 127.20.231.129:37587)
0619 17:02:25.029945 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:25.029949 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:25.030637 (+ 688us) server_negotiation.cc:407] Connection header received
0619 17:02:25.030679 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:25.030682 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:25.030699 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:25.030764 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:02:25.030984 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.032020 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.033297 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.033987 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.034062 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:25.034116 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:25.034471 (+ 355us) server_negotiation.cc:299] Negotiation successful
0619 17:02:25.034519 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":92,"threads_started":1}
I20260619 17:02:25.034605 8336 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06eba101256f436a8a83a13983faf2b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:25.034361 8550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:25.029274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:40275 (local address 127.0.0.1:43294)
0619 17:02:25.029992 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:25.030020 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:25.030074 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:25.030369 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:25.030370 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:25.030372 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:02:25.030414 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:25.030415 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.031773 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.031774 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:25.033002 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:25.033004 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.034087 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.034088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:25.034162 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:25.034167 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 17:02:25.034239 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 17:02:25.034285 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":701,"thread_start_us":94,"threads_started":1}
I20260619 17:02:25.034819 8547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:25.029498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:37587 (local address 127.0.0.1:51238)
0619 17:02:25.030525 (+ 1027us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:25.030535 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:25.030583 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:25.030842 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:25.030843 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:25.030845 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:02:25.030907 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:25.030909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.032132 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.032133 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:25.033202 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:25.033204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.034152 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.034153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:25.034248 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:25.034253 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 17:02:25.034312 (+ 59us) client_negotiation.cc:240] Negotiation successful
0619 17:02:25.034355 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1015,"thread_start_us":73,"threads_started":1}
I20260619 17:02:25.035559 8202 tablet_service.cc:1467] Processing CreateTablet for tablet 06eba101256f436a8a83a13983faf2b2 (DEFAULT_TABLE table=test-table [id=ce4c24e1264f4672bf0e10fa2b564214]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 17:02:25.035692 8470 tablet_service.cc:1467] Processing CreateTablet for tablet 06eba101256f436a8a83a13983faf2b2 (DEFAULT_TABLE table=test-table [id=ce4c24e1264f4672bf0e10fa2b564214]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 17:02:25.035924 8202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06eba101256f436a8a83a13983faf2b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:25.035975 8470 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06eba101256f436a8a83a13983faf2b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:25.038040 8553 tablet_bootstrap.cc:492] T 06eba101256f436a8a83a13983faf2b2 P 4ad9430654f243c3968b4e6607266dc2: Bootstrap starting.
I20260619 17:02:25.038784 8553 tablet_bootstrap.cc:654] T 06eba101256f436a8a83a13983faf2b2 P 4ad9430654f243c3968b4e6607266dc2: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:25.039342 8554 tablet_bootstrap.cc:492] T 06eba101256f436a8a83a13983faf2b2 P 429ac84ef650434f9725b23b3424c90f: Bootstrap starting.
I20260619 17:02:25.039465 8553 log.cc:824] T 06eba101256f436a8a83a13983faf2b2 P 4ad9430654f243c3968b4e6607266dc2: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:25.040114 8554 tablet_bootstrap.cc:654] T 06eba101256f436a8a83a13983faf2b2 P 429ac84ef650434f9725b23b3424c90f: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:25.040328 8553 tablet_bootstrap.cc:492] T 06eba101256f436a8a83a13983faf2b2 P 4ad9430654f243c3968b4e6607266dc2: No bootstrap required, opened a new log
I20260619 17:02:25.040412 8553 ts_tablet_manager.cc:1397] T 06eba101256f436a8a83a13983faf2b2 P 4ad9430654f243c3968b4e6607266dc2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 17:02:25.040788 8554 log.cc:824] T 06eba101256f436a8a83a13983faf2b2 P 429ac84ef650434f9725b23b3424c90f: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:25.041163 8555 tablet_bootstrap.cc:492] T 06eba101256f436a8a83a13983faf2b2 P c4c55ccad53545b7a1cc12d429468642: Bootstrap starting.
I20260619 17:02:25.041633 8554 tablet_bootstrap.cc:492] T 06eba101256f436a8a83a13983faf2b2 P 429ac84ef650434f9725b23b3424c90f: No bootstrap required, opened a new log
I20260619 17:02:25.041726 8554 ts_tablet_manager.cc:1397] T 06eba101256f436a8a83a13983faf2b2 P 429ac84ef650434f9725b23b3424c90f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 17:02:25.041867 8555 tablet_bootstrap.cc:654] T 06eba101256f436a8a83a13983faf2b2 P c4c55ccad53545b7a1cc12d429468642: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:25.042467 8555 log.cc:824] T 06eba101256f436a8a83a13983faf2b2 P c4c55ccad53545b7a1cc12d429468642: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:25.043057 8553 raft_consensus.cc:348] T 06eba101256f436a8a83a13983faf2b2 P 4ad9430654f243c3968b4e6607266dc2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad9430654f243c3968b4e6607266dc2" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 32969 } } peers { permanent_uuid: "429ac84ef650434f9725b23b3424c90f" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40275 } } peers { permanent_uuid: "c4c55ccad53545b7a1cc12d429468642" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 37587 } }
I20260619 17:02:25.043273 8553 raft_consensus.cc:374] T 06eba101256f436a8a83a13983faf2b2 P 4ad9430654f243c3968b4e6607266dc2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:25.043322 8553 raft_consensus.cc:729] T 06eba101256f436a8a83a13983faf2b2 P 4ad9430654f243c3968b4e6607266dc2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ad9430654f243c3968b4e6607266dc2, State: Initialized, Role: FOLLOWER
I20260619 17:02:25.043497 8555 tablet_bootstrap.cc:492] T 06eba101256f436a8a83a13983faf2b2 P c4c55ccad53545b7a1cc12d429468642: No bootstrap required, opened a new log
I20260619 17:02:25.043490 8553 consensus_queue.cc:260] T 06eba101256f436a8a83a13983faf2b2 P 4ad9430654f243c3968b4e6607266dc2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad9430654f243c3968b4e6607266dc2" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 32969 } } peers { permanent_uuid: "429ac84ef650434f9725b23b3424c90f" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40275 } } peers { permanent_uuid: "c4c55ccad53545b7a1cc12d429468642" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 37587 } }
I20260619 17:02:25.043599 8555 ts_tablet_manager.cc:1397] T 06eba101256f436a8a83a13983faf2b2 P c4c55ccad53545b7a1cc12d429468642: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 17:02:25.043816 8553 ts_tablet_manager.cc:1428] T 06eba101256f436a8a83a13983faf2b2 P 4ad9430654f243c3968b4e6607266dc2: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260619 17:02:25.043963 8402 heartbeater.cc:503] Master 127.20.231.190:39035 was elected leader, sending a full tablet report...
I20260619 17:02:25.043849 8554 raft_consensus.cc:348] T 06eba101256f436a8a83a13983faf2b2 P 429ac84ef650434f9725b23b3424c90f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad9430654f243c3968b4e6607266dc2" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 32969 } } peers { permanent_uuid: "429ac84ef650434f9725b23b3424c90f" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40275 } } peers { permanent_uuid: "c4c55ccad53545b7a1cc12d429468642" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 37587 } }
I20260619 17:02:25.044071 8554 raft_consensus.cc:374] T 06eba101256f436a8a83a13983faf2b2 P 429ac84ef650434f9725b23b3424c90f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:25.044122 8554 raft_consensus.cc:729] T 06eba101256f436a8a83a13983faf2b2 P 429ac84ef650434f9725b23b3424c90f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 429ac84ef650434f9725b23b3424c90f, State: Initialized, Role: FOLLOWER
I20260619 17:02:25.044263 8554 consensus_queue.cc:260] T 06eba101256f436a8a83a13983faf2b2 P 429ac84ef650434f9725b23b3424c90f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad9430654f243c3968b4e6607266dc2" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 32969 } } peers { permanent_uuid: "429ac84ef650434f9725b23b3424c90f" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40275 } } peers { permanent_uuid: "c4c55ccad53545b7a1cc12d429468642" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 37587 } }
I20260619 17:02:25.044564 8554 ts_tablet_manager.cc:1428] T 06eba101256f436a8a83a13983faf2b2 P 429ac84ef650434f9725b23b3424c90f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 17:02:25.044601 8536 heartbeater.cc:503] Master 127.20.231.190:39035 was elected leader, sending a full tablet report...
I20260619 17:02:25.045975 8555 raft_consensus.cc:348] T 06eba101256f436a8a83a13983faf2b2 P c4c55ccad53545b7a1cc12d429468642 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad9430654f243c3968b4e6607266dc2" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 32969 } } peers { permanent_uuid: "429ac84ef650434f9725b23b3424c90f" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40275 } } peers { permanent_uuid: "c4c55ccad53545b7a1cc12d429468642" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 37587 } }
I20260619 17:02:25.046120 8555 raft_consensus.cc:374] T 06eba101256f436a8a83a13983faf2b2 P c4c55ccad53545b7a1cc12d429468642 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:25.046144 8555 raft_consensus.cc:729] T 06eba101256f436a8a83a13983faf2b2 P c4c55ccad53545b7a1cc12d429468642 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4c55ccad53545b7a1cc12d429468642, State: Initialized, Role: FOLLOWER
I20260619 17:02:25.046265 8555 consensus_queue.cc:260] T 06eba101256f436a8a83a13983faf2b2 P c4c55ccad53545b7a1cc12d429468642 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad9430654f243c3968b4e6607266dc2" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 32969 } } peers { permanent_uuid: "429ac84ef650434f9725b23b3424c90f" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40275 } } peers { permanent_uuid: "c4c55ccad53545b7a1cc12d429468642" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 37587 } }
I20260619 17:02:25.046528 8555 ts_tablet_manager.cc:1428] T 06eba101256f436a8a83a13983faf2b2 P c4c55ccad53545b7a1cc12d429468642: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 17:02:25.046572 8268 heartbeater.cc:503] Master 127.20.231.190:39035 was elected leader, sending a full tablet report...
I20260619 17:02:25.058681 8561 raft_consensus.cc:482] T 06eba101256f436a8a83a13983faf2b2 P c4c55ccad53545b7a1cc12d429468642 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 17:02:25.058812 8561 raft_consensus.cc:504] T 06eba101256f436a8a83a13983faf2b2 P c4c55ccad53545b7a1cc12d429468642 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad9430654f243c3968b4e6607266dc2" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 32969 } } peers { permanent_uuid: "429ac84ef650434f9725b23b3424c90f" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40275 } } peers { permanent_uuid: "c4c55ccad53545b7a1cc12d429468642" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 37587 } }
I20260619 17:02:25.059333 8561 leader_election.cc:288] T 06eba101256f436a8a83a13983faf2b2 P c4c55ccad53545b7a1cc12d429468642 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4ad9430654f243c3968b4e6607266dc2 (127.20.231.130:32969), 429ac84ef650434f9725b23b3424c90f (127.20.231.131:40275)
I20260619 17:02:25.062973 8270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:25.059560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:32969 (local address 127.20.231.129:53163)
0619 17:02:25.059622 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:25.059635 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:25.059678 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:25.059873 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:25.059874 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:25.059877 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:02:25.059962 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:25.059964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.060955 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.060956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:25.061830 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:25.061833 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.062591 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.062592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:25.062689 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:25.062694 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 17:02:25.062780 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 17:02:25.062834 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 17:02:25.063207 8551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:25.059658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:53163 (local address 127.20.231.130:32969)
0619 17:02:25.059706 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:25.059708 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:25.059717 (+ 9us) server_negotiation.cc:407] Connection header received
0619 17:02:25.059747 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:25.059748 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:25.059760 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:25.059819 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:02:25.060104 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.060824 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.061944 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.062475 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.062540 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:25.062549 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:25.063055 (+ 506us) server_negotiation.cc:299] Negotiation successful
0619 17:02:25.063110 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 17:02:25.063377 8562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:25.059560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:40275 (local address 127.20.231.129:34233)
0619 17:02:25.060124 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:25.060136 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:25.060198 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:25.060311 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:25.060312 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:25.060314 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:02:25.060387 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:25.060389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.061148 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.061149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:25.062067 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:25.062070 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.062817 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.062819 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:25.063116 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:25.063121 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 17:02:25.063236 (+ 115us) client_negotiation.cc:240] Negotiation successful
0619 17:02:25.063284 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"mutex_wait_us":294,"thread_start_us":146,"threads_started":1}
I20260619 17:02:25.063454 8549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:25.059684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:34233 (local address 127.20.231.131:40275)
0619 17:02:25.059734 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:25.059736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:25.060204 (+ 468us) server_negotiation.cc:407] Connection header received
0619 17:02:25.060254 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:25.060255 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:25.060263 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:25.060308 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:02:25.060481 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.061057 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.062188 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.062702 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.062791 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:25.062800 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:25.063267 (+ 467us) server_negotiation.cc:299] Negotiation successful
0619 17:02:25.063329 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 17:02:25.063565 8356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06eba101256f436a8a83a13983faf2b2" candidate_uuid: "c4c55ccad53545b7a1cc12d429468642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ad9430654f243c3968b4e6607266dc2" is_pre_election: true
I20260619 17:02:25.063776 8356 raft_consensus.cc:2454] T 06eba101256f436a8a83a13983faf2b2 P 4ad9430654f243c3968b4e6607266dc2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4c55ccad53545b7a1cc12d429468642 in term 0.
I20260619 17:02:25.063794 8490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06eba101256f436a8a83a13983faf2b2" candidate_uuid: "c4c55ccad53545b7a1cc12d429468642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "429ac84ef650434f9725b23b3424c90f" is_pre_election: true
I20260619 17:02:25.064021 8490 raft_consensus.cc:2454] T 06eba101256f436a8a83a13983faf2b2 P 429ac84ef650434f9725b23b3424c90f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4c55ccad53545b7a1cc12d429468642 in term 0.
I20260619 17:02:25.064045 8157 leader_election.cc:302] T 06eba101256f436a8a83a13983faf2b2 P c4c55ccad53545b7a1cc12d429468642 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ad9430654f243c3968b4e6607266dc2, c4c55ccad53545b7a1cc12d429468642; no voters:
I20260619 17:02:25.064172 8561 raft_consensus.cc:2781] T 06eba101256f436a8a83a13983faf2b2 P c4c55ccad53545b7a1cc12d429468642 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 17:02:25.064234 8561 raft_consensus.cc:482] T 06eba101256f436a8a83a13983faf2b2 P c4c55ccad53545b7a1cc12d429468642 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 17:02:25.064260 8561 raft_consensus.cc:3037] T 06eba101256f436a8a83a13983faf2b2 P c4c55ccad53545b7a1cc12d429468642 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:25.064980 8561 raft_consensus.cc:504] T 06eba101256f436a8a83a13983faf2b2 P c4c55ccad53545b7a1cc12d429468642 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad9430654f243c3968b4e6607266dc2" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 32969 } } peers { permanent_uuid: "429ac84ef650434f9725b23b3424c90f" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40275 } } peers { permanent_uuid: "c4c55ccad53545b7a1cc12d429468642" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 37587 } }
I20260619 17:02:25.065131 8561 leader_election.cc:288] T 06eba101256f436a8a83a13983faf2b2 P c4c55ccad53545b7a1cc12d429468642 [CANDIDATE]: Term 1 election: Requested vote from peers 4ad9430654f243c3968b4e6607266dc2 (127.20.231.130:32969), 429ac84ef650434f9725b23b3424c90f (127.20.231.131:40275)
I20260619 17:02:25.065287 8356 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06eba101256f436a8a83a13983faf2b2" candidate_uuid: "c4c55ccad53545b7a1cc12d429468642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ad9430654f243c3968b4e6607266dc2"
I20260619 17:02:25.065407 8356 raft_consensus.cc:3037] T 06eba101256f436a8a83a13983faf2b2 P 4ad9430654f243c3968b4e6607266dc2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:25.065448 8490 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06eba101256f436a8a83a13983faf2b2" candidate_uuid: "c4c55ccad53545b7a1cc12d429468642" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "429ac84ef650434f9725b23b3424c90f"
I20260619 17:02:25.065562 8490 raft_consensus.cc:3037] T 06eba101256f436a8a83a13983faf2b2 P 429ac84ef650434f9725b23b3424c90f [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:25.066279 8356 raft_consensus.cc:2454] T 06eba101256f436a8a83a13983faf2b2 P 4ad9430654f243c3968b4e6607266dc2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4c55ccad53545b7a1cc12d429468642 in term 1.
I20260619 17:02:25.066416 8490 raft_consensus.cc:2454] T 06eba101256f436a8a83a13983faf2b2 P 429ac84ef650434f9725b23b3424c90f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4c55ccad53545b7a1cc12d429468642 in term 1.
I20260619 17:02:25.066520 8157 leader_election.cc:302] T 06eba101256f436a8a83a13983faf2b2 P c4c55ccad53545b7a1cc12d429468642 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ad9430654f243c3968b4e6607266dc2, c4c55ccad53545b7a1cc12d429468642; no voters:
I20260619 17:02:25.066617 8561 raft_consensus.cc:2781] T 06eba101256f436a8a83a13983faf2b2 P c4c55ccad53545b7a1cc12d429468642 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:02:25.066810 8561 raft_consensus.cc:686] T 06eba101256f436a8a83a13983faf2b2 P c4c55ccad53545b7a1cc12d429468642 [term 1 LEADER]: Becoming Leader. State: Replica: c4c55ccad53545b7a1cc12d429468642, State: Running, Role: LEADER
I20260619 17:02:25.066944 8561 consensus_queue.cc:237] T 06eba101256f436a8a83a13983faf2b2 P c4c55ccad53545b7a1cc12d429468642 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad9430654f243c3968b4e6607266dc2" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 32969 } } peers { permanent_uuid: "429ac84ef650434f9725b23b3424c90f" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40275 } } peers { permanent_uuid: "c4c55ccad53545b7a1cc12d429468642" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 37587 } }
I20260619 17:02:25.067835 8077 catalog_manager.cc:5495] T 06eba101256f436a8a83a13983faf2b2 P c4c55ccad53545b7a1cc12d429468642 reported cstate change: term changed from 0 to 1, leader changed from <none> to c4c55ccad53545b7a1cc12d429468642 (127.20.231.129). New cstate: current_term: 1 leader_uuid: "c4c55ccad53545b7a1cc12d429468642" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ad9430654f243c3968b4e6607266dc2" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 32969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "429ac84ef650434f9725b23b3424c90f" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4c55ccad53545b7a1cc12d429468642" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 37587 } health_report { overall_health: HEALTHY } } }
W20260619 17:02:25.077831 8403 tablet.cc:2369] T 06eba101256f436a8a83a13983faf2b2 P 4ad9430654f243c3968b4e6607266dc2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 17:02:25.082528 8552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:25.080037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51242 (local address 127.20.231.129:37587)
0619 17:02:25.080115 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:25.080117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:25.080124 (+ 7us) server_negotiation.cc:407] Connection header received
0619 17:02:25.080160 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:25.080161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:25.080181 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:25.080260 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 17:02:25.080393 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.080870 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.081666 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.081870 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.081948 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:25.082128 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 17:02:25.082243 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 17:02:25.082361 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:25.082381 (+ 20us) server_negotiation.cc:299] Negotiation successful
0619 17:02:25.082404 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 17:02:25.084753 8490 raft_consensus.cc:1261] T 06eba101256f436a8a83a13983faf2b2 P 429ac84ef650434f9725b23b3424c90f [term 1 FOLLOWER]: Refusing update from remote peer c4c55ccad53545b7a1cc12d429468642: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 17:02:25.084753 8356 raft_consensus.cc:1261] T 06eba101256f436a8a83a13983faf2b2 P 4ad9430654f243c3968b4e6607266dc2 [term 1 FOLLOWER]: Refusing update from remote peer c4c55ccad53545b7a1cc12d429468642: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 17:02:25.085208 8563 consensus_queue.cc:1035] T 06eba101256f436a8a83a13983faf2b2 P c4c55ccad53545b7a1cc12d429468642 [LEADER]: Connected to new peer: Peer: permanent_uuid: "429ac84ef650434f9725b23b3424c90f" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 40275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 17:02:25.085366 8561 consensus_queue.cc:1035] T 06eba101256f436a8a83a13983faf2b2 P c4c55ccad53545b7a1cc12d429468642 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ad9430654f243c3968b4e6607266dc2" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 32969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 17:02:25.088554 8567 mvcc.cc:205] Tried to move back new op lower bound from 7298615480663633920 to 7298615480594886656. Current Snapshot: MvccSnapshot[applied={T|T < 7298615480663633920 or (T in {7298615480663633920})}]
I20260619 17:02:25.090953 8076 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 17:02:25.091079 8076 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 17:02:25.091892 8076 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2e56a3932c054215afa01ef7ecf386e1: Sending DeleteTablet for 3 replicas of tablet 06eba101256f436a8a83a13983faf2b2
I20260619 17:02:25.092324 8470 tablet_service.cc:1514] Processing DeleteTablet for tablet 06eba101256f436a8a83a13983faf2b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:02:25 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:43294
I20260619 17:02:25.092314 8336 tablet_service.cc:1514] Processing DeleteTablet for tablet 06eba101256f436a8a83a13983faf2b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:02:25 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:41608
I20260619 17:02:25.092626 8202 tablet_service.cc:1514] Processing DeleteTablet for tablet 06eba101256f436a8a83a13983faf2b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:02:25 UTC) from {username='kudu', principal='kudu/127.20.231.190@KRBTEST.COM'} at 127.0.0.1:51238
I20260619 17:02:25.092664 8575 tablet_replica.cc:330] T 06eba101256f436a8a83a13983faf2b2 P 4ad9430654f243c3968b4e6607266dc2: stopping tablet replica
I20260619 17:02:25.092772 8575 raft_consensus.cc:2229] T 06eba101256f436a8a83a13983faf2b2 P 4ad9430654f243c3968b4e6607266dc2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 17:02:25.092819 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 8139
I20260619 17:02:25.092856 8577 tablet_replica.cc:330] T 06eba101256f436a8a83a13983faf2b2 P c4c55ccad53545b7a1cc12d429468642: stopping tablet replica
I20260619 17:02:25.093828 8576 tablet_replica.cc:330] T 06eba101256f436a8a83a13983faf2b2 P 429ac84ef650434f9725b23b3424c90f: stopping tablet replica
I20260619 17:02:25.093928 8576 raft_consensus.cc:2229] T 06eba101256f436a8a83a13983faf2b2 P 429ac84ef650434f9725b23b3424c90f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 17:02:25.094025 8576 raft_consensus.cc:2258] T 06eba101256f436a8a83a13983faf2b2 P 429ac84ef650434f9725b23b3424c90f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 17:02:25.092896 8575 raft_consensus.cc:2258] T 06eba101256f436a8a83a13983faf2b2 P 4ad9430654f243c3968b4e6607266dc2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 17:02:25.095511 8576 ts_tablet_manager.cc:1905] T 06eba101256f436a8a83a13983faf2b2 P 429ac84ef650434f9725b23b3424c90f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 17:02:25.095592 8575 ts_tablet_manager.cc:1905] T 06eba101256f436a8a83a13983faf2b2 P 4ad9430654f243c3968b4e6607266dc2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 17:02:25.097124 8576 ts_tablet_manager.cc:1918] T 06eba101256f436a8a83a13983faf2b2 P 429ac84ef650434f9725b23b3424c90f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 17:02:25.097158 8575 ts_tablet_manager.cc:1918] T 06eba101256f436a8a83a13983faf2b2 P 4ad9430654f243c3968b4e6607266dc2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 17:02:25.097203 8576 log.cc:1196] T 06eba101256f436a8a83a13983faf2b2 P 429ac84ef650434f9725b23b3424c90f: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-2/wal/wals/06eba101256f436a8a83a13983faf2b2
I20260619 17:02:25.097249 8575 log.cc:1196] T 06eba101256f436a8a83a13983faf2b2 P 4ad9430654f243c3968b4e6607266dc2: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781888301345144-21406-0/minicluster-data/ts-1/wal/wals/06eba101256f436a8a83a13983faf2b2
I20260619 17:02:25.097483 8576 ts_tablet_manager.cc:1939] T 06eba101256f436a8a83a13983faf2b2 P 429ac84ef650434f9725b23b3424c90f: Deleting consensus metadata
I20260619 17:02:25.097556 8575 ts_tablet_manager.cc:1939] T 06eba101256f436a8a83a13983faf2b2 P 4ad9430654f243c3968b4e6607266dc2: Deleting consensus metadata
I20260619 17:02:25.098029 8063 catalog_manager.cc:4841] TS 4ad9430654f243c3968b4e6607266dc2 (127.20.231.130:32969): tablet 06eba101256f436a8a83a13983faf2b2 (table test-table [id=ce4c24e1264f4672bf0e10fa2b564214]) successfully deleted
I20260619 17:02:25.098035 8062 catalog_manager.cc:4841] TS 429ac84ef650434f9725b23b3424c90f (127.20.231.131:40275): tablet 06eba101256f436a8a83a13983faf2b2 (table test-table [id=ce4c24e1264f4672bf0e10fa2b564214]) successfully deleted
W20260619 17:02:25.098325 8063 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.231.129:37587 (error 108)
I20260619 17:02:25.098591 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 8273
I20260619 17:02:25.099370 8547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:25.098775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:37587 (local address 127.0.0.1:51248)
0619 17:02:25.099286 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:25.099326 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.129:37587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":490}
W20260619 17:02:25.099536 8063 catalog_manager.cc:4584] TS c4c55ccad53545b7a1cc12d429468642 (127.20.231.129:37587): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 06eba101256f436a8a83a13983faf2b2: Network error: Client connection negotiation failed: client connection to 127.20.231.129:37587: connect: Connection refused (error 111)
I20260619 17:02:25.103638 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 8407
I20260619 17:02:25.108470 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 8045
2026-06-19T17:02:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2021 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-06-19T17:02:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T17:02:25Z Disabled control of system clock
I20260619 17:02:25.137238 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:42391
--webserver_interface=127.20.231.190
--webserver_port=0
--builtin_ntp_servers=127.20.231.148:43449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:42391
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 17:02:25.221328 8582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:25.221578 8582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:25.221601 8582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:25.222929 8582 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:02:25.222981 8582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:25.222994 8582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:25.223007 8582 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:02:25.223019 8582 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:02:25.224460 8582 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:43449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:42391
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:42391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:25.224740 8582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:25.225078 8582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:25.230365 8587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:25.230448 8582 server_base.cc:1034] running on GCE node
W20260619 17:02:25.230404 8588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:25.230379 8590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:25.230957 8582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:25.231271 8582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:25.232429 8582 hybrid_clock.cc:648] HybridClock initialized: now 1781888545232417 us; error 35 us; skew 500 ppm
I20260619 17:02:25.233800 8582 webserver.cc:460] Webserver started at http://127.20.231.190:41261/ using document root <none> and password file <none>
I20260619 17:02:25.234053 8582 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:25.234126 8582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:25.234318 8582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:25.235548 8582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "9127f084aa27427ca1b2c93f5eae2d36"
format_stamp: "Formatted at 2026-06-19 17:02:25 on dist-test-slave-vnbj"
server_key: "f53d68f05c1e570ea5a08b89d7b498af"
server_key_iv: "648f46a622c376285d40eee50b3542f9"
server_key_version: "encryptionkey@0"
I20260619 17:02:25.235918 8582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "9127f084aa27427ca1b2c93f5eae2d36"
format_stamp: "Formatted at 2026-06-19 17:02:25 on dist-test-slave-vnbj"
server_key: "f53d68f05c1e570ea5a08b89d7b498af"
server_key_iv: "648f46a622c376285d40eee50b3542f9"
server_key_version: "encryptionkey@0"
I20260619 17:02:25.237308 8582 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 17:02:25.238234 8596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:25.238478 8582 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 17:02:25.238591 8582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "9127f084aa27427ca1b2c93f5eae2d36"
format_stamp: "Formatted at 2026-06-19 17:02:25 on dist-test-slave-vnbj"
server_key: "f53d68f05c1e570ea5a08b89d7b498af"
server_key_iv: "648f46a622c376285d40eee50b3542f9"
server_key_version: "encryptionkey@0"
I20260619 17:02:25.238683 8582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:25.252156 8582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:25.252573 8582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:25.252748 8582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:25.257138 8582 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:42391
I20260619 17:02:25.257198 8648 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:42391 every 8 connection(s)
I20260619 17:02:25.257608 8582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 17:02:25.258641 8649 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:25.261322 8649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9127f084aa27427ca1b2c93f5eae2d36: Bootstrap starting.
I20260619 17:02:25.261962 8649 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9127f084aa27427ca1b2c93f5eae2d36: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:25.262311 8649 log.cc:824] T 00000000000000000000000000000000 P 9127f084aa27427ca1b2c93f5eae2d36: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:25.262732 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 8582
I20260619 17:02:25.262883 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 17:02:25.263080 21406 external_mini_cluster.cc:1428] Setting key df1742da76347d248f8aa1a3fd9eb285
I20260619 17:02:25.263334 8649 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9127f084aa27427ca1b2c93f5eae2d36: No bootstrap required, opened a new log
I20260619 17:02:25.265295 8649 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9127f084aa27427ca1b2c93f5eae2d36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9127f084aa27427ca1b2c93f5eae2d36" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 42391 } }
I20260619 17:02:25.265498 8649 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9127f084aa27427ca1b2c93f5eae2d36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:25.265548 8649 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9127f084aa27427ca1b2c93f5eae2d36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9127f084aa27427ca1b2c93f5eae2d36, State: Initialized, Role: FOLLOWER
I20260619 17:02:25.265704 8649 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9127f084aa27427ca1b2c93f5eae2d36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9127f084aa27427ca1b2c93f5eae2d36" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 42391 } }
I20260619 17:02:25.265806 8649 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9127f084aa27427ca1b2c93f5eae2d36 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 17:02:25.265856 8649 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9127f084aa27427ca1b2c93f5eae2d36 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 17:02:25.265905 8649 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9127f084aa27427ca1b2c93f5eae2d36 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:25.266804 8649 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9127f084aa27427ca1b2c93f5eae2d36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9127f084aa27427ca1b2c93f5eae2d36" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 42391 } }
I20260619 17:02:25.266985 8649 leader_election.cc:302] T 00000000000000000000000000000000 P 9127f084aa27427ca1b2c93f5eae2d36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9127f084aa27427ca1b2c93f5eae2d36; no voters:
I20260619 17:02:25.267215 8649 leader_election.cc:288] T 00000000000000000000000000000000 P 9127f084aa27427ca1b2c93f5eae2d36 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 17:02:25.267279 8654 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9127f084aa27427ca1b2c93f5eae2d36 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:02:25.267397 8654 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9127f084aa27427ca1b2c93f5eae2d36 [term 1 LEADER]: Becoming Leader. State: Replica: 9127f084aa27427ca1b2c93f5eae2d36, State: Running, Role: LEADER
I20260619 17:02:25.267500 8649 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9127f084aa27427ca1b2c93f5eae2d36 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:02:25.267478 8654 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9127f084aa27427ca1b2c93f5eae2d36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9127f084aa27427ca1b2c93f5eae2d36" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 42391 } }
I20260619 17:02:25.267824 8654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9127f084aa27427ca1b2c93f5eae2d36 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9127f084aa27427ca1b2c93f5eae2d36. Latest consensus state: current_term: 1 leader_uuid: "9127f084aa27427ca1b2c93f5eae2d36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9127f084aa27427ca1b2c93f5eae2d36" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 42391 } } }
I20260619 17:02:25.267910 8654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9127f084aa27427ca1b2c93f5eae2d36 [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:25.267838 8656 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9127f084aa27427ca1b2c93f5eae2d36 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9127f084aa27427ca1b2c93f5eae2d36" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9127f084aa27427ca1b2c93f5eae2d36" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 42391 } } }
I20260619 17:02:25.268090 8656 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9127f084aa27427ca1b2c93f5eae2d36 [sys.catalog]: This master's current role is: LEADER
W20260619 17:02:25.269281 8669 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9127f084aa27427ca1b2c93f5eae2d36: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 17:02:25.269340 8669 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 17:02:25.269690 8659 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 17:02:25.269906 8659 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 17:02:25.271121 8653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:25.263620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59482 (local address 127.20.231.190:42391)
0619 17:02:25.263894 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:25.263910 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:25.263936 (+ 26us) server_negotiation.cc:407] Connection header received
0619 17:02:25.265291 (+ 1355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:25.265301 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:25.265377 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:25.265588 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:25.265831 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.267096 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.268266 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.268951 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.269093 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:25.270316 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:25.270320 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:25.270320 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:25.270534 (+ 214us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:25.270578 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:25.270593 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:25.270662 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:25.270792 (+ 130us) server_negotiation.cc:299] Negotiation successful
0619 17:02:25.270903 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":107,"threads_started":1}
I20260619 17:02:25.271683 8659 catalog_manager.cc:1347] Generated new cluster ID: 2ce0517caeb9498fa9a07910074a5198
I20260619 17:02:25.271744 8659 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 17:02:25.278864 8659 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 17:02:25.279446 8659 catalog_manager.cc:1504] Loading token signing keys...
I20260619 17:02:25.308240 8659 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9127f084aa27427ca1b2c93f5eae2d36: Generated new TSK 0
I20260619 17:02:25.308519 8659 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 17:02:25.314356 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.129:0
--local_ip_for_outbound_sockets=127.20.231.129
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:42391
--builtin_ntp_servers=127.20.231.148:43449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 17:02:25.397890 8673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:25.398123 8673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:25.398146 8673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:25.399477 8673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:25.399523 8673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:25.399549 8673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.129
I20260619 17:02:25.400938 8673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:43449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.20.231.129
--webserver_port=0
--tserver_master_addrs=127.20.231.190:42391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.129
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:25.401211 8673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:25.401527 8673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:25.407298 8681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:25.407270 8678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:25.407536 8679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:25.407574 8673 server_base.cc:1034] running on GCE node
I20260619 17:02:25.407868 8673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:25.408154 8673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:25.409363 8673 hybrid_clock.cc:648] HybridClock initialized: now 1781888545409345 us; error 28 us; skew 500 ppm
I20260619 17:02:25.410679 8673 webserver.cc:460] Webserver started at http://127.20.231.129:33667/ using document root <none> and password file <none>
I20260619 17:02:25.411010 8673 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:25.411087 8673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:25.411229 8673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:25.412503 8673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-0/data/instance:
uuid: "dea0f3e780564420bbe0c4d6279fd71a"
format_stamp: "Formatted at 2026-06-19 17:02:25 on dist-test-slave-vnbj"
server_key: "b387a16b6e6e687537a328a22fbcdc12"
server_key_iv: "29563ab765f449191a182acca053845b"
server_key_version: "encryptionkey@0"
I20260619 17:02:25.412886 8673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance:
uuid: "dea0f3e780564420bbe0c4d6279fd71a"
format_stamp: "Formatted at 2026-06-19 17:02:25 on dist-test-slave-vnbj"
server_key: "b387a16b6e6e687537a328a22fbcdc12"
server_key_iv: "29563ab765f449191a182acca053845b"
server_key_version: "encryptionkey@0"
I20260619 17:02:25.414288 8673 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 17:02:25.415138 8687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:25.415385 8673 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 17:02:25.415500 8673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-0/wal
uuid: "dea0f3e780564420bbe0c4d6279fd71a"
format_stamp: "Formatted at 2026-06-19 17:02:25 on dist-test-slave-vnbj"
server_key: "b387a16b6e6e687537a328a22fbcdc12"
server_key_iv: "29563ab765f449191a182acca053845b"
server_key_version: "encryptionkey@0"
I20260619 17:02:25.415603 8673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:25.429999 8673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:25.430404 8673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:25.430586 8673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:25.430943 8673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:02:25.431360 8673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:02:25.431425 8673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:25.431464 8673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:02:25.431493 8673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:25.437693 8673 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.129:39643
I20260619 17:02:25.437739 8800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.129:39643 every 8 connection(s)
I20260619 17:02:25.438138 8673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-0/data/info.pb
I20260619 17:02:25.439497 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 8673
I20260619 17:02:25.439594 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-0/wal/instance
I20260619 17:02:25.439806 21406 external_mini_cluster.cc:1428] Setting key 99ad8b414444425f1d8902880596f638
I20260619 17:02:25.441043 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.130:0
--local_ip_for_outbound_sockets=127.20.231.130
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:42391
--builtin_ntp_servers=127.20.231.148:43449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 17:02:25.444957 8653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:25.439111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.129:57779 (local address 127.20.231.190:42391)
0619 17:02:25.439188 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:25.439191 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:25.439435 (+ 244us) server_negotiation.cc:407] Connection header received
0619 17:02:25.440195 (+ 760us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:25.440196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:25.440213 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:25.440282 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:25.441326 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.442036 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.443311 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.443574 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.443641 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:25.444619 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:25.444620 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:25.444621 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:25.444625 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:25.444671 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:25.444677 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:25.444748 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:25.444856 (+ 108us) server_negotiation.cc:299] Negotiation successful
0619 17:02:25.444891 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 17:02:25.445305 8803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:25.439080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:42391 (local address 127.20.231.129:57779)
0619 17:02:25.439279 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:25.439325 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:25.440026 (+ 701us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:25.440382 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:25.440386 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:25.440394 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:25.441141 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:25.441149 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.442140 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.442141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:25.443224 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:25.443227 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.443677 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.443678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:25.443848 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:25.444452 (+ 604us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 17:02:25.444453 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:25.444494 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 17:02:25.444524 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:25.444758 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:25.444766 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:25.444769 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 17:02:25.444853 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 17:02:25.444982 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":89,"threads_started":1}
I20260619 17:02:25.446035 8801 heartbeater.cc:347] Connected to a master server at 127.20.231.190:42391
I20260619 17:02:25.446151 8801 heartbeater.cc:464] Registering TS with master...
I20260619 17:02:25.446380 8801 heartbeater.cc:511] Master 127.20.231.190:42391 requested a full tablet report, sending...
I20260619 17:02:25.447027 8613 ts_manager.cc:194] Registered new tserver with Master: dea0f3e780564420bbe0c4d6279fd71a (127.20.231.129:39643)
I20260619 17:02:25.447665 8613 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.231.129:57779
W20260619 17:02:25.526358 8804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:25.526590 8804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:25.526614 8804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:25.527927 8804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:25.527980 8804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:25.528008 8804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.130
I20260619 17:02:25.529495 8804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:43449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.20.231.130
--webserver_port=0
--tserver_master_addrs=127.20.231.190:42391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.130
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:25.529791 8804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:25.530125 8804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 17:02:25.535687 8804 server_base.cc:1034] running on GCE node
W20260619 17:02:25.535674 8812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:25.535670 8810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:25.535704 8809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:25.536293 8804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:25.536571 8804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:25.537781 8804 hybrid_clock.cc:648] HybridClock initialized: now 1781888545537766 us; error 33 us; skew 500 ppm
I20260619 17:02:25.539430 8804 webserver.cc:460] Webserver started at http://127.20.231.130:42317/ using document root <none> and password file <none>
I20260619 17:02:25.539691 8804 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:25.539763 8804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:25.539930 8804 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:25.541069 8804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-1/data/instance:
uuid: "d2f28d483b754932bd58cb8727ef45b9"
format_stamp: "Formatted at 2026-06-19 17:02:25 on dist-test-slave-vnbj"
server_key: "354605d3aa8f8f667f3e12bfc887a2e3"
server_key_iv: "3f09fd62a24d77bc119020d84d70dadc"
server_key_version: "encryptionkey@0"
I20260619 17:02:25.541447 8804 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance:
uuid: "d2f28d483b754932bd58cb8727ef45b9"
format_stamp: "Formatted at 2026-06-19 17:02:25 on dist-test-slave-vnbj"
server_key: "354605d3aa8f8f667f3e12bfc887a2e3"
server_key_iv: "3f09fd62a24d77bc119020d84d70dadc"
server_key_version: "encryptionkey@0"
I20260619 17:02:25.543040 8804 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260619 17:02:25.544001 8818 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:25.544230 8804 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:25.544354 8804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-1/wal
uuid: "d2f28d483b754932bd58cb8727ef45b9"
format_stamp: "Formatted at 2026-06-19 17:02:25 on dist-test-slave-vnbj"
server_key: "354605d3aa8f8f667f3e12bfc887a2e3"
server_key_iv: "3f09fd62a24d77bc119020d84d70dadc"
server_key_version: "encryptionkey@0"
I20260619 17:02:25.544454 8804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:25.561945 8804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:25.562337 8804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:25.562511 8804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:25.562781 8804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:02:25.563180 8804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:02:25.563241 8804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:25.563277 8804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:02:25.563306 8804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:25.570379 8804 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.130:45921
I20260619 17:02:25.570426 8931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.130:45921 every 8 connection(s)
I20260619 17:02:25.570811 8804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-1/data/info.pb
I20260619 17:02:25.576768 8653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:25.571698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:55843 (local address 127.20.231.190:42391)
0619 17:02:25.571795 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:25.571797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:25.572153 (+ 356us) server_negotiation.cc:407] Connection header received
0619 17:02:25.572737 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:25.572738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:25.572750 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:25.572825 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:25.573619 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.574159 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.575281 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.575480 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.575541 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:25.576485 (+ 944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:25.576487 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:25.576487 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:25.576492 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:25.576509 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:25.576512 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:25.576559 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:25.576674 (+ 115us) server_negotiation.cc:299] Negotiation successful
0619 17:02:25.576702 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 17:02:25.577004 8934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:25.571698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:42391 (local address 127.20.231.130:55843)
0619 17:02:25.572015 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:25.572054 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:25.572583 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:25.572905 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:25.572909 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:25.572917 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:25.573513 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:25.573518 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.574257 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.574258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:25.575201 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:25.575204 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.575565 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.575566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:25.575692 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:25.576293 (+ 601us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 17:02:25.576294 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:25.576353 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 17:02:25.576389 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:25.576592 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:25.576598 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:25.576601 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 17:02:25.576663 (+ 62us) client_negotiation.cc:240] Negotiation successful
0619 17:02:25.576756 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":176,"threads_started":1}
I20260619 17:02:25.577095 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 8804
I20260619 17:02:25.577198 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-1/wal/instance
I20260619 17:02:25.577415 21406 external_mini_cluster.cc:1428] Setting key 1f6c2ff980a5a54c55143895e2ad88c9
I20260619 17:02:25.577673 8932 heartbeater.cc:347] Connected to a master server at 127.20.231.190:42391
I20260619 17:02:25.577790 8932 heartbeater.cc:464] Registering TS with master...
I20260619 17:02:25.577987 8932 heartbeater.cc:511] Master 127.20.231.190:42391 requested a full tablet report, sending...
I20260619 17:02:25.578328 8613 ts_manager.cc:194] Registered new tserver with Master: d2f28d483b754932bd58cb8727ef45b9 (127.20.231.130:45921)
I20260619 17:02:25.578541 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.20.231.131:0
--local_ip_for_outbound_sockets=127.20.231.131
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:42391
--builtin_ntp_servers=127.20.231.148:43449
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 17:02:25.578796 8613 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.231.130:55843
W20260619 17:02:25.664871 8935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:25.665102 8935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:25.665125 8935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:25.666275 8935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:25.666317 8935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:25.666343 8935 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.20.231.131
I20260619 17:02:25.667945 8935 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:43449
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.20.231.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.20.231.131
--webserver_port=0
--tserver_master_addrs=127.20.231.190:42391
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.20.231.131
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:25.668227 8935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:25.668514 8935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:25.674018 8941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:25.674048 8935 server_base.cc:1034] running on GCE node
W20260619 17:02:25.674005 8940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:25.674005 8943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:25.674578 8935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:25.674926 8935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:25.676124 8935 hybrid_clock.cc:648] HybridClock initialized: now 1781888545676097 us; error 47 us; skew 500 ppm
I20260619 17:02:25.677564 8935 webserver.cc:460] Webserver started at http://127.20.231.131:41993/ using document root <none> and password file <none>
I20260619 17:02:25.677914 8935 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:25.677999 8935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:25.678153 8935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:25.679364 8935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-2/data/instance:
uuid: "ccd3499d3a8745719843f37e0d9199d1"
format_stamp: "Formatted at 2026-06-19 17:02:25 on dist-test-slave-vnbj"
server_key: "6f697f5759895990579f05049bb2308f"
server_key_iv: "f57118e59e3e48ead612e82ac28a5920"
server_key_version: "encryptionkey@0"
I20260619 17:02:25.679735 8935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance:
uuid: "ccd3499d3a8745719843f37e0d9199d1"
format_stamp: "Formatted at 2026-06-19 17:02:25 on dist-test-slave-vnbj"
server_key: "6f697f5759895990579f05049bb2308f"
server_key_iv: "f57118e59e3e48ead612e82ac28a5920"
server_key_version: "encryptionkey@0"
I20260619 17:02:25.681291 8935 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:25.682222 8949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:25.682444 8935 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:25.682549 8935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-2/wal
uuid: "ccd3499d3a8745719843f37e0d9199d1"
format_stamp: "Formatted at 2026-06-19 17:02:25 on dist-test-slave-vnbj"
server_key: "6f697f5759895990579f05049bb2308f"
server_key_iv: "f57118e59e3e48ead612e82ac28a5920"
server_key_version: "encryptionkey@0"
I20260619 17:02:25.682627 8935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:25.695197 8935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:25.695537 8935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:25.695698 8935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:25.695935 8935 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 17:02:25.696297 8935 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 17:02:25.696357 8935 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:25.696398 8935 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 17:02:25.696426 8935 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:25.702292 8935 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.131:44961
I20260619 17:02:25.702346 9062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.131:44961 every 8 connection(s)
I20260619 17:02:25.702713 8935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-2/data/info.pb
I20260619 17:02:25.703926 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 8935
I20260619 17:02:25.704038 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-2/wal/instance
I20260619 17:02:25.704241 21406 external_mini_cluster.cc:1428] Setting key 4543557d73a373ba7db52f2eb1981aa5
I20260619 17:02:25.709076 8653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:25.703546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.131:55263 (local address 127.20.231.190:42391)
0619 17:02:25.703625 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:25.703628 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:25.703891 (+ 263us) server_negotiation.cc:407] Connection header received
0619 17:02:25.704770 (+ 879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:25.704771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:25.704785 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:25.704852 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:25.705820 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.706327 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.707495 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.707697 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.707765 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:25.708750 (+ 985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:25.708751 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:25.708752 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:25.708758 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:25.708779 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:25.708784 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:25.708847 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:25.708976 (+ 129us) server_negotiation.cc:299] Negotiation successful
0619 17:02:25.709011 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 17:02:25.709236 9065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:25.703480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:42391 (local address 127.20.231.131:55263)
0619 17:02:25.703679 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:25.703745 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:25.704532 (+ 787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:25.705013 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:25.705017 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:25.705041 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:25.705720 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:25.705726 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.706418 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.706419 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:25.707394 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:25.707397 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.707780 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.707781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:25.707962 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:25.708587 (+ 625us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 17:02:25.708588 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:25.708618 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 17:02:25.708643 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:25.708857 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:25.708861 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:25.708864 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 17:02:25.708921 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 17:02:25.709015 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":109,"threads_started":1}
I20260619 17:02:25.709933 9063 heartbeater.cc:347] Connected to a master server at 127.20.231.190:42391
I20260619 17:02:25.710044 9063 heartbeater.cc:464] Registering TS with master...
I20260619 17:02:25.710259 9063 heartbeater.cc:511] Master 127.20.231.190:42391 requested a full tablet report, sending...
I20260619 17:02:25.710741 8613 ts_manager.cc:194] Registered new tserver with Master: ccd3499d3a8745719843f37e0d9199d1 (127.20.231.131:44961)
I20260619 17:02:25.711140 8613 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.20.231.131:55263
I20260619 17:02:25.716655 21406 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 17:02:25.721884 8653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:25.718415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59494 (local address 127.20.231.190:42391)
0619 17:02:25.718498 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:25.718500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:25.718557 (+ 57us) server_negotiation.cc:407] Connection header received
0619 17:02:25.718609 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:25.718610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:25.718622 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:25.718674 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:25.718940 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.719525 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.720399 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.720587 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.720646 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:25.721642 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:25.721644 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:25.721644 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:25.721649 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:25.721664 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:25.721668 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:25.721715 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:25.721792 (+ 77us) server_negotiation.cc:299] Negotiation successful
0619 17:02:25.721821 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 17:02:25.723829 8613 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59494:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 17:02:25.725925 8613 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 17:02:25.732945 9075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:25.728781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53556 (local address 127.20.231.130:45921)
0619 17:02:25.728973 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:25.728978 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:25.729032 (+ 54us) server_negotiation.cc:407] Connection header received
0619 17:02:25.729124 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:25.729126 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:25.729142 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:25.729229 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:02:25.729447 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.730630 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.731929 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.732501 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.732605 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:25.732697 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:25.732808 (+ 111us) server_negotiation.cc:299] Negotiation successful
0619 17:02:25.732848 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":83,"threads_started":1}
I20260619 17:02:25.732968 9072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:25.728523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.130:45921 (local address 127.0.0.1:53556)
0619 17:02:25.728979 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:25.728987 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:25.729036 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:25.729260 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:25.729263 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:25.729265 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:02:25.729356 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:25.729359 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.730717 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.730718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:25.731866 (+ 1148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:25.731868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.732607 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.732608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:25.732694 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:25.732710 (+ 16us) client_negotiation.cc:769] Sending connection context
0619 17:02:25.732790 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 17:02:25.732849 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"mutex_wait_us":70,"thread_start_us":100,"threads_started":1}
I20260619 17:02:25.733291 9071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:25.728512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:39643 (local address 127.0.0.1:53060)
0619 17:02:25.728718 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:25.728769 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:25.728847 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:25.729296 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:25.729297 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:25.729302 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:02:25.729359 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:25.729374 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.730918 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.730919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:25.732009 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:25.732011 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.733059 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.733061 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:25.733141 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:25.733145 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 17:02:25.733195 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 17:02:25.733230 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"mutex_wait_us":12,"thread_start_us":84,"threads_started":1}
I20260619 17:02:25.733291 9076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:25.728875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49734 (local address 127.20.231.131:44961)
0619 17:02:25.729072 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:25.729075 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:25.729084 (+ 9us) server_negotiation.cc:407] Connection header received
0619 17:02:25.729123 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:25.729125 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:25.729141 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:25.729229 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:02:25.729491 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.730296 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.731770 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.732531 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.732606 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:25.732697 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:25.733158 (+ 461us) server_negotiation.cc:299] Negotiation successful
0619 17:02:25.733208 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":90,"threads_started":1}
I20260619 17:02:25.733675 9074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:25.728597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53060 (local address 127.20.231.129:39643)
0619 17:02:25.728777 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:25.728783 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:25.728938 (+ 155us) server_negotiation.cc:407] Connection header received
0619 17:02:25.728991 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:25.728994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:25.729012 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:25.729089 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:02:25.729445 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.730627 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.732103 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.732883 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.732946 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:25.733023 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:25.733465 (+ 442us) server_negotiation.cc:299] Negotiation successful
0619 17:02:25.733567 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":77,"threads_started":1}
I20260619 17:02:25.733723 8866 tablet_service.cc:1467] Processing CreateTablet for tablet 6d7cf0a64b6144a386830a0db3b6d2a9 (DEFAULT_TABLE table=test-table [id=a94e7d1495e34322a32e47503f1ce144]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 17:02:25.734123 8866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d7cf0a64b6144a386830a0db3b6d2a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:25.734304 8735 tablet_service.cc:1467] Processing CreateTablet for tablet 6d7cf0a64b6144a386830a0db3b6d2a9 (DEFAULT_TABLE table=test-table [id=a94e7d1495e34322a32e47503f1ce144]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 17:02:25.734606 8735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d7cf0a64b6144a386830a0db3b6d2a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:25.737116 9077 tablet_bootstrap.cc:492] T 6d7cf0a64b6144a386830a0db3b6d2a9 P d2f28d483b754932bd58cb8727ef45b9: Bootstrap starting.
I20260619 17:02:25.737201 9073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:25.728523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:44961 (local address 127.0.0.1:49734)
0619 17:02:25.728842 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:25.728852 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:25.728902 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:25.729296 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:25.729297 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:25.729300 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:02:25.729391 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:25.729392 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.730383 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.730384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:25.731671 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:25.731673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.732734 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.732736 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:25.732800 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:25.732803 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 17:02:25.732857 (+ 54us) client_negotiation.cc:240] Negotiation successful
0619 17:02:25.732905 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":3,"thread_start_us":101,"threads_started":1}
I20260619 17:02:25.737787 9077 tablet_bootstrap.cc:654] T 6d7cf0a64b6144a386830a0db3b6d2a9 P d2f28d483b754932bd58cb8727ef45b9: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:25.738142 9077 log.cc:824] T 6d7cf0a64b6144a386830a0db3b6d2a9 P d2f28d483b754932bd58cb8727ef45b9: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:25.738090 8997 tablet_service.cc:1467] Processing CreateTablet for tablet 6d7cf0a64b6144a386830a0db3b6d2a9 (DEFAULT_TABLE table=test-table [id=a94e7d1495e34322a32e47503f1ce144]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 17:02:25.738400 8997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d7cf0a64b6144a386830a0db3b6d2a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:25.738528 9078 tablet_bootstrap.cc:492] T 6d7cf0a64b6144a386830a0db3b6d2a9 P dea0f3e780564420bbe0c4d6279fd71a: Bootstrap starting.
I20260619 17:02:25.739042 9077 tablet_bootstrap.cc:492] T 6d7cf0a64b6144a386830a0db3b6d2a9 P d2f28d483b754932bd58cb8727ef45b9: No bootstrap required, opened a new log
I20260619 17:02:25.739123 9077 ts_tablet_manager.cc:1397] T 6d7cf0a64b6144a386830a0db3b6d2a9 P d2f28d483b754932bd58cb8727ef45b9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 17:02:25.739199 9078 tablet_bootstrap.cc:654] T 6d7cf0a64b6144a386830a0db3b6d2a9 P dea0f3e780564420bbe0c4d6279fd71a: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:25.739480 9078 log.cc:824] T 6d7cf0a64b6144a386830a0db3b6d2a9 P dea0f3e780564420bbe0c4d6279fd71a: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:25.740319 9078 tablet_bootstrap.cc:492] T 6d7cf0a64b6144a386830a0db3b6d2a9 P dea0f3e780564420bbe0c4d6279fd71a: No bootstrap required, opened a new log
I20260619 17:02:25.740403 9078 ts_tablet_manager.cc:1397] T 6d7cf0a64b6144a386830a0db3b6d2a9 P dea0f3e780564420bbe0c4d6279fd71a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 17:02:25.742017 9081 tablet_bootstrap.cc:492] T 6d7cf0a64b6144a386830a0db3b6d2a9 P ccd3499d3a8745719843f37e0d9199d1: Bootstrap starting.
I20260619 17:02:25.741848 9077 raft_consensus.cc:348] T 6d7cf0a64b6144a386830a0db3b6d2a9 P d2f28d483b754932bd58cb8727ef45b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea0f3e780564420bbe0c4d6279fd71a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 39643 } } peers { permanent_uuid: "d2f28d483b754932bd58cb8727ef45b9" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45921 } } peers { permanent_uuid: "ccd3499d3a8745719843f37e0d9199d1" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 44961 } }
I20260619 17:02:25.742163 9077 raft_consensus.cc:374] T 6d7cf0a64b6144a386830a0db3b6d2a9 P d2f28d483b754932bd58cb8727ef45b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:25.742223 9077 raft_consensus.cc:729] T 6d7cf0a64b6144a386830a0db3b6d2a9 P d2f28d483b754932bd58cb8727ef45b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2f28d483b754932bd58cb8727ef45b9, State: Initialized, Role: FOLLOWER
I20260619 17:02:25.742342 9077 consensus_queue.cc:260] T 6d7cf0a64b6144a386830a0db3b6d2a9 P d2f28d483b754932bd58cb8727ef45b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea0f3e780564420bbe0c4d6279fd71a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 39643 } } peers { permanent_uuid: "d2f28d483b754932bd58cb8727ef45b9" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45921 } } peers { permanent_uuid: "ccd3499d3a8745719843f37e0d9199d1" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 44961 } }
I20260619 17:02:25.742650 8932 heartbeater.cc:503] Master 127.20.231.190:42391 was elected leader, sending a full tablet report...
I20260619 17:02:25.742818 9081 tablet_bootstrap.cc:654] T 6d7cf0a64b6144a386830a0db3b6d2a9 P ccd3499d3a8745719843f37e0d9199d1: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:25.742802 9078 raft_consensus.cc:348] T 6d7cf0a64b6144a386830a0db3b6d2a9 P dea0f3e780564420bbe0c4d6279fd71a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea0f3e780564420bbe0c4d6279fd71a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 39643 } } peers { permanent_uuid: "d2f28d483b754932bd58cb8727ef45b9" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45921 } } peers { permanent_uuid: "ccd3499d3a8745719843f37e0d9199d1" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 44961 } }
I20260619 17:02:25.743037 9078 raft_consensus.cc:374] T 6d7cf0a64b6144a386830a0db3b6d2a9 P dea0f3e780564420bbe0c4d6279fd71a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:25.743073 9078 raft_consensus.cc:729] T 6d7cf0a64b6144a386830a0db3b6d2a9 P dea0f3e780564420bbe0c4d6279fd71a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dea0f3e780564420bbe0c4d6279fd71a, State: Initialized, Role: FOLLOWER
I20260619 17:02:25.743160 9081 log.cc:824] T 6d7cf0a64b6144a386830a0db3b6d2a9 P ccd3499d3a8745719843f37e0d9199d1: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:25.743212 9078 consensus_queue.cc:260] T 6d7cf0a64b6144a386830a0db3b6d2a9 P dea0f3e780564420bbe0c4d6279fd71a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea0f3e780564420bbe0c4d6279fd71a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 39643 } } peers { permanent_uuid: "d2f28d483b754932bd58cb8727ef45b9" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45921 } } peers { permanent_uuid: "ccd3499d3a8745719843f37e0d9199d1" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 44961 } }
I20260619 17:02:25.743485 9078 ts_tablet_manager.cc:1428] T 6d7cf0a64b6144a386830a0db3b6d2a9 P dea0f3e780564420bbe0c4d6279fd71a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260619 17:02:25.743547 9077 ts_tablet_manager.cc:1428] T 6d7cf0a64b6144a386830a0db3b6d2a9 P d2f28d483b754932bd58cb8727ef45b9: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260619 17:02:25.743610 8801 heartbeater.cc:503] Master 127.20.231.190:42391 was elected leader, sending a full tablet report...
I20260619 17:02:25.743932 9081 tablet_bootstrap.cc:492] T 6d7cf0a64b6144a386830a0db3b6d2a9 P ccd3499d3a8745719843f37e0d9199d1: No bootstrap required, opened a new log
I20260619 17:02:25.743991 9081 ts_tablet_manager.cc:1397] T 6d7cf0a64b6144a386830a0db3b6d2a9 P ccd3499d3a8745719843f37e0d9199d1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 17:02:25.745949 9081 raft_consensus.cc:348] T 6d7cf0a64b6144a386830a0db3b6d2a9 P ccd3499d3a8745719843f37e0d9199d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea0f3e780564420bbe0c4d6279fd71a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 39643 } } peers { permanent_uuid: "d2f28d483b754932bd58cb8727ef45b9" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45921 } } peers { permanent_uuid: "ccd3499d3a8745719843f37e0d9199d1" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 44961 } }
I20260619 17:02:25.746068 9081 raft_consensus.cc:374] T 6d7cf0a64b6144a386830a0db3b6d2a9 P ccd3499d3a8745719843f37e0d9199d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:25.746088 9081 raft_consensus.cc:729] T 6d7cf0a64b6144a386830a0db3b6d2a9 P ccd3499d3a8745719843f37e0d9199d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccd3499d3a8745719843f37e0d9199d1, State: Initialized, Role: FOLLOWER
I20260619 17:02:25.746208 9081 consensus_queue.cc:260] T 6d7cf0a64b6144a386830a0db3b6d2a9 P ccd3499d3a8745719843f37e0d9199d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea0f3e780564420bbe0c4d6279fd71a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 39643 } } peers { permanent_uuid: "d2f28d483b754932bd58cb8727ef45b9" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45921 } } peers { permanent_uuid: "ccd3499d3a8745719843f37e0d9199d1" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 44961 } }
I20260619 17:02:25.746447 9081 ts_tablet_manager.cc:1428] T 6d7cf0a64b6144a386830a0db3b6d2a9 P ccd3499d3a8745719843f37e0d9199d1: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260619 17:02:25.746557 9063 heartbeater.cc:503] Master 127.20.231.190:42391 was elected leader, sending a full tablet report...
W20260619 17:02:25.821689 8933 tablet.cc:2369] T 6d7cf0a64b6144a386830a0db3b6d2a9 P d2f28d483b754932bd58cb8727ef45b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 17:02:25.839078 9082 raft_consensus.cc:482] T 6d7cf0a64b6144a386830a0db3b6d2a9 P d2f28d483b754932bd58cb8727ef45b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 17:02:25.839293 9082 raft_consensus.cc:504] T 6d7cf0a64b6144a386830a0db3b6d2a9 P d2f28d483b754932bd58cb8727ef45b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea0f3e780564420bbe0c4d6279fd71a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 39643 } } peers { permanent_uuid: "d2f28d483b754932bd58cb8727ef45b9" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45921 } } peers { permanent_uuid: "ccd3499d3a8745719843f37e0d9199d1" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 44961 } }
I20260619 17:02:25.839743 9082 leader_election.cc:288] T 6d7cf0a64b6144a386830a0db3b6d2a9 P d2f28d483b754932bd58cb8727ef45b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dea0f3e780564420bbe0c4d6279fd71a (127.20.231.129:39643), ccd3499d3a8745719843f37e0d9199d1 (127.20.231.131:44961)
I20260619 17:02:25.843053 9076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:25.840103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:39125 (local address 127.20.231.131:44961)
0619 17:02:25.840146 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:25.840149 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:25.840160 (+ 11us) server_negotiation.cc:407] Connection header received
0619 17:02:25.840220 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:25.840221 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:25.840229 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:25.840358 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:02:25.840536 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.841019 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.842224 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.842626 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.842730 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:25.842740 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:25.842892 (+ 152us) server_negotiation.cc:299] Negotiation successful
0619 17:02:25.842937 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 17:02:25.843053 8934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:25.839984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.131:44961 (local address 127.20.231.130:39125)
0619 17:02:25.840063 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:25.840079 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:25.840150 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:25.840367 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:25.840368 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:25.840370 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:02:25.840450 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:25.840453 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.841109 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.841110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:25.842123 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:25.842126 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.842715 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.842716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:25.842790 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:25.842802 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 17:02:25.842885 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 17:02:25.842937 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260619 17:02:25.843169 9074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:25.840096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.20.231.130:44197 (local address 127.20.231.129:39643)
0619 17:02:25.840145 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:25.840148 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:25.840339 (+ 191us) server_negotiation.cc:407] Connection header received
0619 17:02:25.840413 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:25.840414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:25.840425 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:25.840511 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 17:02:25.840666 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.841257 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.842289 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.842774 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.842875 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:25.842883 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:25.843061 (+ 178us) server_negotiation.cc:299] Negotiation successful
0619 17:02:25.843103 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260619 17:02:25.843178 9086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:25.839983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:39643 (local address 127.20.231.130:44197)
0619 17:02:25.840261 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:25.840270 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:25.840353 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:25.840517 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:25.840518 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:25.840519 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 17:02:25.840593 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:25.840595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.841338 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.841340 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:25.842175 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:25.842177 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.842880 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.842881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:25.842968 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:25.842971 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 17:02:25.843052 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 17:02:25.843093 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":80,"thread_start_us":124,"threads_started":1}
I20260619 17:02:25.843616 8755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d7cf0a64b6144a386830a0db3b6d2a9" candidate_uuid: "d2f28d483b754932bd58cb8727ef45b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dea0f3e780564420bbe0c4d6279fd71a" is_pre_election: true
I20260619 17:02:25.843611 9017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d7cf0a64b6144a386830a0db3b6d2a9" candidate_uuid: "d2f28d483b754932bd58cb8727ef45b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccd3499d3a8745719843f37e0d9199d1" is_pre_election: true
I20260619 17:02:25.843782 8755 raft_consensus.cc:2454] T 6d7cf0a64b6144a386830a0db3b6d2a9 P dea0f3e780564420bbe0c4d6279fd71a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2f28d483b754932bd58cb8727ef45b9 in term 0.
I20260619 17:02:25.843783 9017 raft_consensus.cc:2454] T 6d7cf0a64b6144a386830a0db3b6d2a9 P ccd3499d3a8745719843f37e0d9199d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d2f28d483b754932bd58cb8727ef45b9 in term 0.
I20260619 17:02:25.844028 8819 leader_election.cc:302] T 6d7cf0a64b6144a386830a0db3b6d2a9 P d2f28d483b754932bd58cb8727ef45b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ccd3499d3a8745719843f37e0d9199d1, d2f28d483b754932bd58cb8727ef45b9; no voters:
I20260619 17:02:25.844158 9082 raft_consensus.cc:2781] T 6d7cf0a64b6144a386830a0db3b6d2a9 P d2f28d483b754932bd58cb8727ef45b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 17:02:25.844220 9082 raft_consensus.cc:482] T 6d7cf0a64b6144a386830a0db3b6d2a9 P d2f28d483b754932bd58cb8727ef45b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 17:02:25.844286 9082 raft_consensus.cc:3037] T 6d7cf0a64b6144a386830a0db3b6d2a9 P d2f28d483b754932bd58cb8727ef45b9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:25.845148 9082 raft_consensus.cc:504] T 6d7cf0a64b6144a386830a0db3b6d2a9 P d2f28d483b754932bd58cb8727ef45b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea0f3e780564420bbe0c4d6279fd71a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 39643 } } peers { permanent_uuid: "d2f28d483b754932bd58cb8727ef45b9" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45921 } } peers { permanent_uuid: "ccd3499d3a8745719843f37e0d9199d1" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 44961 } }
I20260619 17:02:25.845337 9082 leader_election.cc:288] T 6d7cf0a64b6144a386830a0db3b6d2a9 P d2f28d483b754932bd58cb8727ef45b9 [CANDIDATE]: Term 1 election: Requested vote from peers dea0f3e780564420bbe0c4d6279fd71a (127.20.231.129:39643), ccd3499d3a8745719843f37e0d9199d1 (127.20.231.131:44961)
I20260619 17:02:25.845515 9017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d7cf0a64b6144a386830a0db3b6d2a9" candidate_uuid: "d2f28d483b754932bd58cb8727ef45b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ccd3499d3a8745719843f37e0d9199d1"
I20260619 17:02:25.845525 8755 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d7cf0a64b6144a386830a0db3b6d2a9" candidate_uuid: "d2f28d483b754932bd58cb8727ef45b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dea0f3e780564420bbe0c4d6279fd71a"
I20260619 17:02:25.845599 8755 raft_consensus.cc:3037] T 6d7cf0a64b6144a386830a0db3b6d2a9 P dea0f3e780564420bbe0c4d6279fd71a [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:25.845629 9017 raft_consensus.cc:3037] T 6d7cf0a64b6144a386830a0db3b6d2a9 P ccd3499d3a8745719843f37e0d9199d1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:25.846385 9017 raft_consensus.cc:2454] T 6d7cf0a64b6144a386830a0db3b6d2a9 P ccd3499d3a8745719843f37e0d9199d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2f28d483b754932bd58cb8727ef45b9 in term 1.
I20260619 17:02:25.846412 8755 raft_consensus.cc:2454] T 6d7cf0a64b6144a386830a0db3b6d2a9 P dea0f3e780564420bbe0c4d6279fd71a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d2f28d483b754932bd58cb8727ef45b9 in term 1.
I20260619 17:02:25.846606 8821 leader_election.cc:302] T 6d7cf0a64b6144a386830a0db3b6d2a9 P d2f28d483b754932bd58cb8727ef45b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d2f28d483b754932bd58cb8727ef45b9, dea0f3e780564420bbe0c4d6279fd71a; no voters:
I20260619 17:02:25.846720 9082 raft_consensus.cc:2781] T 6d7cf0a64b6144a386830a0db3b6d2a9 P d2f28d483b754932bd58cb8727ef45b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:02:25.846932 9082 raft_consensus.cc:686] T 6d7cf0a64b6144a386830a0db3b6d2a9 P d2f28d483b754932bd58cb8727ef45b9 [term 1 LEADER]: Becoming Leader. State: Replica: d2f28d483b754932bd58cb8727ef45b9, State: Running, Role: LEADER
I20260619 17:02:25.847076 9082 consensus_queue.cc:237] T 6d7cf0a64b6144a386830a0db3b6d2a9 P d2f28d483b754932bd58cb8727ef45b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea0f3e780564420bbe0c4d6279fd71a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 39643 } } peers { permanent_uuid: "d2f28d483b754932bd58cb8727ef45b9" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45921 } } peers { permanent_uuid: "ccd3499d3a8745719843f37e0d9199d1" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 44961 } }
I20260619 17:02:25.848070 8612 catalog_manager.cc:5495] T 6d7cf0a64b6144a386830a0db3b6d2a9 P d2f28d483b754932bd58cb8727ef45b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to d2f28d483b754932bd58cb8727ef45b9 (127.20.231.130). New cstate: current_term: 1 leader_uuid: "d2f28d483b754932bd58cb8727ef45b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dea0f3e780564420bbe0c4d6279fd71a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 39643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d2f28d483b754932bd58cb8727ef45b9" member_type: VOTER last_known_addr { host: "127.20.231.130" port: 45921 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ccd3499d3a8745719843f37e0d9199d1" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 44961 } health_report { overall_health: UNKNOWN } } }
I20260619 17:02:25.879384 9075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:25.876408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53560 (local address 127.20.231.130:45921)
0619 17:02:25.876501 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:25.876503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:25.876513 (+ 10us) server_negotiation.cc:407] Connection header received
0619 17:02:25.876557 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:25.876558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:25.876579 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:25.876649 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 17:02:25.876879 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.877358 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.878265 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:25.878473 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:25.878541 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:25.878767 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 17:02:25.878991 (+ 224us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 17:02:25.879091 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:25.879130 (+ 39us) server_negotiation.cc:299] Negotiation successful
0619 17:02:25.879153 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 17:02:25.881600 9017 raft_consensus.cc:1261] T 6d7cf0a64b6144a386830a0db3b6d2a9 P ccd3499d3a8745719843f37e0d9199d1 [term 1 FOLLOWER]: Refusing update from remote peer d2f28d483b754932bd58cb8727ef45b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 17:02:25.881691 8755 raft_consensus.cc:1261] T 6d7cf0a64b6144a386830a0db3b6d2a9 P dea0f3e780564420bbe0c4d6279fd71a [term 1 FOLLOWER]: Refusing update from remote peer d2f28d483b754932bd58cb8727ef45b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 17:02:25.882098 9087 consensus_queue.cc:1035] T 6d7cf0a64b6144a386830a0db3b6d2a9 P d2f28d483b754932bd58cb8727ef45b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dea0f3e780564420bbe0c4d6279fd71a" member_type: VOTER last_known_addr { host: "127.20.231.129" port: 39643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 17:02:25.882277 9082 consensus_queue.cc:1035] T 6d7cf0a64b6144a386830a0db3b6d2a9 P d2f28d483b754932bd58cb8727ef45b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ccd3499d3a8745719843f37e0d9199d1" member_type: VOTER last_known_addr { host: "127.20.231.131" port: 44961 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 17:02:25.887562 8612 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 17:02:25.887691 8612 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59494:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 17:02:25.888526 8612 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9127f084aa27427ca1b2c93f5eae2d36: Sending DeleteTablet for 3 replicas of tablet 6d7cf0a64b6144a386830a0db3b6d2a9
I20260619 17:02:25.888901 8997 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d7cf0a64b6144a386830a0db3b6d2a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:02:25 UTC) from {username='slave'} at 127.0.0.1:49734
I20260619 17:02:25.888916 8866 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d7cf0a64b6144a386830a0db3b6d2a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:02:25 UTC) from {username='slave'} at 127.0.0.1:53556
I20260619 17:02:25.889204 9100 tablet_replica.cc:330] T 6d7cf0a64b6144a386830a0db3b6d2a9 P ccd3499d3a8745719843f37e0d9199d1: stopping tablet replica
I20260619 17:02:25.889262 8735 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d7cf0a64b6144a386830a0db3b6d2a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 17:02:25 UTC) from {username='slave'} at 127.0.0.1:53060
I20260619 17:02:25.889319 9100 raft_consensus.cc:2229] T 6d7cf0a64b6144a386830a0db3b6d2a9 P ccd3499d3a8745719843f37e0d9199d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 17:02:25.889439 9100 raft_consensus.cc:2258] T 6d7cf0a64b6144a386830a0db3b6d2a9 P ccd3499d3a8745719843f37e0d9199d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 17:02:25.889509 9101 tablet_replica.cc:330] T 6d7cf0a64b6144a386830a0db3b6d2a9 P dea0f3e780564420bbe0c4d6279fd71a: stopping tablet replica
I20260619 17:02:25.889611 9101 raft_consensus.cc:2229] T 6d7cf0a64b6144a386830a0db3b6d2a9 P dea0f3e780564420bbe0c4d6279fd71a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 17:02:25.889696 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 8673
I20260619 17:02:25.889722 9101 raft_consensus.cc:2258] T 6d7cf0a64b6144a386830a0db3b6d2a9 P dea0f3e780564420bbe0c4d6279fd71a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 17:02:25.890030 9100 ts_tablet_manager.cc:1905] T 6d7cf0a64b6144a386830a0db3b6d2a9 P ccd3499d3a8745719843f37e0d9199d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 17:02:25.891997 9100 ts_tablet_manager.cc:1918] T 6d7cf0a64b6144a386830a0db3b6d2a9 P ccd3499d3a8745719843f37e0d9199d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 17:02:25.892086 9100 log.cc:1196] T 6d7cf0a64b6144a386830a0db3b6d2a9 P ccd3499d3a8745719843f37e0d9199d1: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-2/wal/wals/6d7cf0a64b6144a386830a0db3b6d2a9
I20260619 17:02:25.892017 9099 tablet_replica.cc:330] T 6d7cf0a64b6144a386830a0db3b6d2a9 P d2f28d483b754932bd58cb8727ef45b9: stopping tablet replica
I20260619 17:02:25.892270 9099 raft_consensus.cc:2229] T 6d7cf0a64b6144a386830a0db3b6d2a9 P d2f28d483b754932bd58cb8727ef45b9 [term 1 LEADER]: Raft consensus shutting down.
I20260619 17:02:25.892370 9100 ts_tablet_manager.cc:1939] T 6d7cf0a64b6144a386830a0db3b6d2a9 P ccd3499d3a8745719843f37e0d9199d1: Deleting consensus metadata
I20260619 17:02:25.892534 9099 raft_consensus.cc:2258] T 6d7cf0a64b6144a386830a0db3b6d2a9 P d2f28d483b754932bd58cb8727ef45b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 17:02:25.893019 9099 ts_tablet_manager.cc:1905] T 6d7cf0a64b6144a386830a0db3b6d2a9 P d2f28d483b754932bd58cb8727ef45b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 17:02:25.893671 8597 catalog_manager.cc:4841] TS ccd3499d3a8745719843f37e0d9199d1 (127.20.231.131:44961): tablet 6d7cf0a64b6144a386830a0db3b6d2a9 (table test-table [id=a94e7d1495e34322a32e47503f1ce144]) successfully deleted
I20260619 17:02:25.895380 9099 ts_tablet_manager.cc:1918] T 6d7cf0a64b6144a386830a0db3b6d2a9 P d2f28d483b754932bd58cb8727ef45b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 17:02:25.895548 9099 log.cc:1196] T 6d7cf0a64b6144a386830a0db3b6d2a9 P d2f28d483b754932bd58cb8727ef45b9: Deleting WAL directory at /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781888301345144-21406-0/minicluster-data/ts-1/wal/wals/6d7cf0a64b6144a386830a0db3b6d2a9
I20260619 17:02:25.895823 9099 ts_tablet_manager.cc:1939] T 6d7cf0a64b6144a386830a0db3b6d2a9 P d2f28d483b754932bd58cb8727ef45b9: Deleting consensus metadata
I20260619 17:02:25.896318 8598 catalog_manager.cc:4841] TS d2f28d483b754932bd58cb8727ef45b9 (127.20.231.130:45921): tablet 6d7cf0a64b6144a386830a0db3b6d2a9 (table test-table [id=a94e7d1495e34322a32e47503f1ce144]) successfully deleted
W20260619 17:02:25.898185 8599 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.20.231.129:39643 (error 108)
I20260619 17:02:25.898358 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 8804
I20260619 17:02:25.899441 9072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:25.899304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.129:39643 (local address 127.0.0.1:53074)
0619 17:02:25.899361 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:25.899402 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.129:39643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260619 17:02:25.899689 8599 catalog_manager.cc:4584] TS dea0f3e780564420bbe0c4d6279fd71a (127.20.231.129:39643): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6d7cf0a64b6144a386830a0db3b6d2a9: Network error: Client connection negotiation failed: client connection to 127.20.231.129:39643: connect: Connection refused (error 111)
I20260619 17:02:25.903378 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 8935
I20260619 17:02:25.908396 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 8582
2026-06-19T17:02:25Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (799 ms)
[----------] 6 tests from EncryptionPolicyTest (8879 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 17:02:25 dist-test-slave-vnbj krb5kdc[9105](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 17:02:25 dist-test-slave-vnbj krb5kdc[9105](info): set up 2 sockets
Jun 19 17:02:25 dist-test-slave-vnbj krb5kdc[9105](info): commencing operation
krb5kdc: starting...
W20260619 17:02:27.059111 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 17:02:27 dist-test-slave-vnbj krb5kdc[9105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T17:02:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T17:02:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 17:02:27.194027 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43369
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43369
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:02:27.278193 9118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:27.278437 9118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:27.278465 9118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:27.279860 9118 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:02:27.279909 9118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:27.279924 9118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:27.279937 9118 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:02:27.279950 9118 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:02:27.281838 9118 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43369
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9118
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:27.282063 9118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:27.282238 9118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:27.287473 9123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:27.287492 9126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:27.287467 9118 server_base.cc:1034] running on GCE node
W20260619 17:02:27.287487 9124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:27.288064 9118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:27.288434 9118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:27.289645 9118 hybrid_clock.cc:648] HybridClock initialized: now 1781888547289644 us; error 51 us; skew 500 ppm
Jun 19 17:02:27 dist-test-slave-vnbj krb5kdc[9105](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888547, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:27.299082 9118 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 17:02:27.299742 9118 webserver.cc:460] Webserver started at http://127.0.0.1:44529/ using document root <none> and password file <none>
I20260619 17:02:27.300105 9118 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:27.300186 9118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:27.300362 9118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:27.301568 9118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "fcb102a6d9e948db89d030b918ad27be"
format_stamp: "Formatted at 2026-06-19 17:02:27 on dist-test-slave-vnbj"
server_key: "d583e95966b999cf36ad17cbbf6d8821"
server_key_iv: "e3ba1245eb35e46c604cd8a858c99ee5"
server_key_version: "encryptionkey@0"
I20260619 17:02:27.301973 9118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "fcb102a6d9e948db89d030b918ad27be"
format_stamp: "Formatted at 2026-06-19 17:02:27 on dist-test-slave-vnbj"
server_key: "d583e95966b999cf36ad17cbbf6d8821"
server_key_iv: "e3ba1245eb35e46c604cd8a858c99ee5"
server_key_version: "encryptionkey@0"
I20260619 17:02:27.303675 9118 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260619 17:02:27.304584 9133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:27.304926 9118 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:27.305073 9118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "fcb102a6d9e948db89d030b918ad27be"
format_stamp: "Formatted at 2026-06-19 17:02:27 on dist-test-slave-vnbj"
server_key: "d583e95966b999cf36ad17cbbf6d8821"
server_key_iv: "e3ba1245eb35e46c604cd8a858c99ee5"
server_key_version: "encryptionkey@0"
I20260619 17:02:27.305166 9118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:27.323632 9118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:27.326545 9118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:27.326740 9118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:27.331583 9118 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43369
I20260619 17:02:27.331624 9185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43369 every 8 connection(s)
I20260619 17:02:27.332067 9118 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 17:02:27.333055 9186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:27.335660 9186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcb102a6d9e948db89d030b918ad27be: Bootstrap starting.
I20260619 17:02:27.336333 9186 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fcb102a6d9e948db89d030b918ad27be: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:27.336661 9186 log.cc:824] T 00000000000000000000000000000000 P fcb102a6d9e948db89d030b918ad27be: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:27.337546 9186 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcb102a6d9e948db89d030b918ad27be: No bootstrap required, opened a new log
I20260619 17:02:27.339520 9186 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcb102a6d9e948db89d030b918ad27be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb102a6d9e948db89d030b918ad27be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43369 } }
I20260619 17:02:27.339675 9186 raft_consensus.cc:374] T 00000000000000000000000000000000 P fcb102a6d9e948db89d030b918ad27be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:27.339701 9186 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcb102a6d9e948db89d030b918ad27be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcb102a6d9e948db89d030b918ad27be, State: Initialized, Role: FOLLOWER
I20260619 17:02:27.339891 9186 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcb102a6d9e948db89d030b918ad27be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb102a6d9e948db89d030b918ad27be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43369 } }
I20260619 17:02:27.340003 9186 raft_consensus.cc:388] T 00000000000000000000000000000000 P fcb102a6d9e948db89d030b918ad27be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 17:02:27.340047 9186 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcb102a6d9e948db89d030b918ad27be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 17:02:27.340098 9186 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcb102a6d9e948db89d030b918ad27be [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:27.340179 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 9118
I20260619 17:02:27.340306 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 17:02:27.340545 21406 external_mini_cluster.cc:1428] Setting key ffa9c3734c93b3e51c873de19547a20b
I20260619 17:02:27.340924 9186 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcb102a6d9e948db89d030b918ad27be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb102a6d9e948db89d030b918ad27be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43369 } }
I20260619 17:02:27.341058 9186 leader_election.cc:302] T 00000000000000000000000000000000 P fcb102a6d9e948db89d030b918ad27be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fcb102a6d9e948db89d030b918ad27be; no voters:
I20260619 17:02:27.341282 9186 leader_election.cc:288] T 00000000000000000000000000000000 P fcb102a6d9e948db89d030b918ad27be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 17:02:27.341362 9190 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcb102a6d9e948db89d030b918ad27be [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:02:27.341534 9190 raft_consensus.cc:686] T 00000000000000000000000000000000 P fcb102a6d9e948db89d030b918ad27be [term 1 LEADER]: Becoming Leader. State: Replica: fcb102a6d9e948db89d030b918ad27be, State: Running, Role: LEADER
I20260619 17:02:27.341599 9186 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcb102a6d9e948db89d030b918ad27be [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:02:27.341634 9190 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcb102a6d9e948db89d030b918ad27be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb102a6d9e948db89d030b918ad27be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43369 } }
I20260619 17:02:27.342131 9192 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcb102a6d9e948db89d030b918ad27be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fcb102a6d9e948db89d030b918ad27be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb102a6d9e948db89d030b918ad27be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43369 } } }
I20260619 17:02:27.342247 9192 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcb102a6d9e948db89d030b918ad27be [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:27.342494 9200 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 17:02:27.342465 9193 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcb102a6d9e948db89d030b918ad27be [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcb102a6d9e948db89d030b918ad27be. Latest consensus state: current_term: 1 leader_uuid: "fcb102a6d9e948db89d030b918ad27be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb102a6d9e948db89d030b918ad27be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43369 } } }
I20260619 17:02:27.342645 9193 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcb102a6d9e948db89d030b918ad27be [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:27.343118 9200 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 17:02:27.344764 9200 catalog_manager.cc:1347] Generated new cluster ID: a41c783d747a4c69a0845460d9fc9cb1
I20260619 17:02:27.344831 9200 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 17:02:27 dist-test-slave-vnbj krb5kdc[9105](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 17:02:27.354748 9191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:27.341144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41075 (local address 127.0.0.1:43369)
0619 17:02:27.341530 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:27.341540 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:27.341577 (+ 37us) server_negotiation.cc:407] Connection header received
0619 17:02:27.341733 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:27.341741 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:27.341826 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:27.341977 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:27.343070 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:27.344814 (+ 1744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:27.346043 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:27.346736 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:27.346890 (+ 154us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:27.350213 (+ 3323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:27.350219 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:27.350219 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:27.350236 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:27.352660 (+ 2424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:27.353213 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:27.353217 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:27.353218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:27.353386 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:27.353655 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:27.353656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:27.353657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:27.354071 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:27.354265 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:27.354368 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 17:02:27.354490 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":110,"threads_started":1}
I20260619 17:02:27.356348 21406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 17:02:27.361280 9200 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 17:02:27.362010 9200 catalog_manager.cc:1504] Loading token signing keys...
I20260619 17:02:27.366452 9191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:27.358085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54029 (local address 127.0.0.1:43369)
0619 17:02:27.358124 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:27.358126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:27.358174 (+ 48us) server_negotiation.cc:407] Connection header received
0619 17:02:27.358257 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:27.358257 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:27.358267 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:27.358323 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:27.359104 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:27.359945 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:27.361119 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:27.361451 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:27.361525 (+ 74us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:27.363191 (+ 1666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:27.363192 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:27.363192 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:27.363200 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:27.364950 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:27.365439 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:27.365440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:27.365440 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:27.365503 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:27.365766 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:27.365767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:27.365767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:27.365899 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:27.366009 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:27.366163 (+ 154us) server_negotiation.cc:299] Negotiation successful
0619 17:02:27.366230 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 17:02:27.368391 9200 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fcb102a6d9e948db89d030b918ad27be: Generated new TSK 0
I20260619 17:02:27.368592 9200 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 17:02:27.372330 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 9118
W20260619 17:02:27.376489 9103 connection.cc:535] client connection to 127.0.0.1:43369 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43369): Connection reset by peer (error 104)
2026-06-19T17:02:27Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1462 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 17:02:27 dist-test-slave-vnbj krb5kdc[9217](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 17:02:27 dist-test-slave-vnbj krb5kdc[9217](info): set up 2 sockets
Jun 19 17:02:27 dist-test-slave-vnbj krb5kdc[9217](info): commencing operation
krb5kdc: starting...
W20260619 17:02:28.439379 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.048s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 17:02:28 dist-test-slave-vnbj krb5kdc[9217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888548, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T17:02:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T17:02:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 17:02:28.575137 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45319
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45319
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:02:28.660277 9230 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:28.660552 9230 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:28.660593 9230 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:28.661836 9230 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:02:28.661916 9230 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:28.661939 9230 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:28.661962 9230 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:02:28.661993 9230 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:02:28.663604 9230 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45319
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45319
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9230
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:28.663916 9230 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:28.664162 9230 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:28.669449 9235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:28.669608 9238 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:28.669608 9236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:28.669857 9230 server_base.cc:1034] running on GCE node
I20260619 17:02:28.670105 9230 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:28.670379 9230 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:28.671530 9230 hybrid_clock.cc:648] HybridClock initialized: now 1781888548671510 us; error 42 us; skew 500 ppm
Jun 19 17:02:28 dist-test-slave-vnbj krb5kdc[9217](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888548, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:28.674667 9230 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 17:02:28.675418 9230 webserver.cc:460] Webserver started at http://127.0.0.1:34301/ using document root <none> and password file <none>
I20260619 17:02:28.675699 9230 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:28.675791 9230 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:28.675962 9230 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:28.677189 9230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "ccdec7de765c48f29141399af021f59b"
format_stamp: "Formatted at 2026-06-19 17:02:28 on dist-test-slave-vnbj"
server_key: "64437b30d1c814f1f4ca6715afaf1ea8"
server_key_iv: "a58dd8a6377673b2a230f08ea0692f3f"
server_key_version: "encryptionkey@0"
I20260619 17:02:28.677599 9230 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "ccdec7de765c48f29141399af021f59b"
format_stamp: "Formatted at 2026-06-19 17:02:28 on dist-test-slave-vnbj"
server_key: "64437b30d1c814f1f4ca6715afaf1ea8"
server_key_iv: "a58dd8a6377673b2a230f08ea0692f3f"
server_key_version: "encryptionkey@0"
I20260619 17:02:28.679373 9230 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.002s
I20260619 17:02:28.680486 9245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:28.680763 9230 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:28.680891 9230 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "ccdec7de765c48f29141399af021f59b"
format_stamp: "Formatted at 2026-06-19 17:02:28 on dist-test-slave-vnbj"
server_key: "64437b30d1c814f1f4ca6715afaf1ea8"
server_key_iv: "a58dd8a6377673b2a230f08ea0692f3f"
server_key_version: "encryptionkey@0"
I20260619 17:02:28.680979 9230 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:28.708707 9230 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:28.711580 9230 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:28.711788 9230 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:28.716311 9230 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45319
I20260619 17:02:28.716349 9297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45319 every 8 connection(s)
I20260619 17:02:28.716790 9230 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 17:02:28.717702 9298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:28.720176 9298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccdec7de765c48f29141399af021f59b: Bootstrap starting.
I20260619 17:02:28.720733 9298 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ccdec7de765c48f29141399af021f59b: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:28.721074 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 9230
I20260619 17:02:28.721102 9298 log.cc:824] T 00000000000000000000000000000000 P ccdec7de765c48f29141399af021f59b: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:28.721204 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 17:02:28.721426 21406 external_mini_cluster.cc:1428] Setting key 4e69511afbe23edbdee04d3f85853482
I20260619 17:02:28.722013 9298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ccdec7de765c48f29141399af021f59b: No bootstrap required, opened a new log
I20260619 17:02:28.724905 9298 raft_consensus.cc:348] T 00000000000000000000000000000000 P ccdec7de765c48f29141399af021f59b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccdec7de765c48f29141399af021f59b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45319 } }
I20260619 17:02:28.725100 9298 raft_consensus.cc:374] T 00000000000000000000000000000000 P ccdec7de765c48f29141399af021f59b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:28.725147 9298 raft_consensus.cc:729] T 00000000000000000000000000000000 P ccdec7de765c48f29141399af021f59b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ccdec7de765c48f29141399af021f59b, State: Initialized, Role: FOLLOWER
I20260619 17:02:28.725284 9298 consensus_queue.cc:260] T 00000000000000000000000000000000 P ccdec7de765c48f29141399af021f59b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccdec7de765c48f29141399af021f59b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45319 } }
I20260619 17:02:28.725379 9298 raft_consensus.cc:388] T 00000000000000000000000000000000 P ccdec7de765c48f29141399af021f59b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 17:02:28.725422 9298 raft_consensus.cc:482] T 00000000000000000000000000000000 P ccdec7de765c48f29141399af021f59b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 17:02:28.725476 9298 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ccdec7de765c48f29141399af021f59b [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:28.726114 9298 raft_consensus.cc:504] T 00000000000000000000000000000000 P ccdec7de765c48f29141399af021f59b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccdec7de765c48f29141399af021f59b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45319 } }
I20260619 17:02:28.726239 9298 leader_election.cc:302] T 00000000000000000000000000000000 P ccdec7de765c48f29141399af021f59b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ccdec7de765c48f29141399af021f59b; no voters:
I20260619 17:02:28.726436 9298 leader_election.cc:288] T 00000000000000000000000000000000 P ccdec7de765c48f29141399af021f59b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 17:02:28.726495 9303 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ccdec7de765c48f29141399af021f59b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:02:28.726653 9303 raft_consensus.cc:686] T 00000000000000000000000000000000 P ccdec7de765c48f29141399af021f59b [term 1 LEADER]: Becoming Leader. State: Replica: ccdec7de765c48f29141399af021f59b, State: Running, Role: LEADER
I20260619 17:02:28.726727 9298 sys_catalog.cc:564] T 00000000000000000000000000000000 P ccdec7de765c48f29141399af021f59b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:02:28.726750 9303 consensus_queue.cc:237] T 00000000000000000000000000000000 P ccdec7de765c48f29141399af021f59b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccdec7de765c48f29141399af021f59b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45319 } }
I20260619 17:02:28.727208 9303 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccdec7de765c48f29141399af021f59b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ccdec7de765c48f29141399af021f59b. Latest consensus state: current_term: 1 leader_uuid: "ccdec7de765c48f29141399af021f59b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccdec7de765c48f29141399af021f59b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45319 } } }
I20260619 17:02:28.727299 9303 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccdec7de765c48f29141399af021f59b [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:28.727475 9304 sys_catalog.cc:455] T 00000000000000000000000000000000 P ccdec7de765c48f29141399af021f59b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ccdec7de765c48f29141399af021f59b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ccdec7de765c48f29141399af021f59b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45319 } } }
I20260619 17:02:28.727558 9304 sys_catalog.cc:458] T 00000000000000000000000000000000 P ccdec7de765c48f29141399af021f59b [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:28.727581 9311 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 17:02:28.728204 9311 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 17:02:28.729059 9302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:28.722083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52559 (local address 127.0.0.1:45319)
0619 17:02:28.722289 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:28.722298 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:28.722327 (+ 29us) server_negotiation.cc:407] Connection header received
0619 17:02:28.723397 (+ 1070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:28.723407 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:28.723482 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:28.723661 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:28.724000 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:28.725055 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:28.726119 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:28.726939 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:28.727096 (+ 157us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:28.727964 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:28.727968 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:28.727969 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:28.728232 (+ 263us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:28.728278 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:28.728297 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:28.728405 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:28.728586 (+ 181us) server_negotiation.cc:299] Negotiation successful
0619 17:02:28.728623 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":94,"threads_started":1}
I20260619 17:02:28.730597 21406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 17:02:28.731230 9311 catalog_manager.cc:1347] Generated new cluster ID: 6c4d2db6ecbd4d33b6d08e88c1854f94
I20260619 17:02:28.731302 9311 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 17:02:28.736948 9302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:28.732434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52319 (local address 127.0.0.1:45319)
0619 17:02:28.732474 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:28.732476 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:28.732601 (+ 125us) server_negotiation.cc:407] Connection header received
0619 17:02:28.732667 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:28.732668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:28.732678 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:28.732747 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:28.732913 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:28.733724 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:28.735076 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:28.735549 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:28.735646 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:28.736620 (+ 974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:28.736621 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:28.736622 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:28.736628 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:28.736649 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:28.736654 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:28.736714 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:28.736842 (+ 128us) server_negotiation.cc:299] Negotiation successful
0619 17:02:28.736855 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 17:02:28.745671 9311 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 17:02:28.746254 9311 catalog_manager.cc:1504] Loading token signing keys...
I20260619 17:02:28.752689 9311 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ccdec7de765c48f29141399af021f59b: Generated new TSK 0
I20260619 17:02:28.753100 9311 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 17:02:28.759332 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 9230
W20260619 17:02:28.763664 9215 connection.cc:535] client connection to 127.0.0.1:45319 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45319): Connection reset by peer (error 104)
2026-06-19T17:02:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1386 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 17:02:28 dist-test-slave-vnbj krb5kdc[9329](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 17:02:28 dist-test-slave-vnbj krb5kdc[9329](info): set up 2 sockets
Jun 19 17:02:28 dist-test-slave-vnbj krb5kdc[9329](info): commencing operation
krb5kdc: starting...
W20260619 17:02:29.811614 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 17:02:29 dist-test-slave-vnbj krb5kdc[9329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T17:02:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T17:02:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 17:02:29.947211 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45767
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45767
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:02:30.032430 9342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:30.032739 9342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:30.032789 9342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:30.034134 9342 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:02:30.034195 9342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:30.034211 9342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:30.034224 9342 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:02:30.034237 9342 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:02:30.035912 9342 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45767
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:30.036201 9342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:30.036463 9342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:30.041963 9348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:30.042029 9347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:30.041975 9350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:30.042165 9342 server_base.cc:1034] running on GCE node
I20260619 17:02:30.042615 9342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:30.042996 9342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:30.044195 9342 hybrid_clock.cc:648] HybridClock initialized: now 1781888550044157 us; error 61 us; skew 500 ppm
Jun 19 17:02:30 dist-test-slave-vnbj krb5kdc[9329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888550, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:30.047282 9342 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 17:02:30.047741 9342 webserver.cc:460] Webserver started at http://127.0.0.1:46085/ using document root <none> and password file <none>
I20260619 17:02:30.047933 9342 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:30.047976 9342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:30.048067 9342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:30.049168 9342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "8a44094d4434494f8d7aad772c830663"
format_stamp: "Formatted at 2026-06-19 17:02:30 on dist-test-slave-vnbj"
server_key: "e8d638d06fe3bc23d9e55d53feb2c4c6"
server_key_iv: "20bed768df5343c5f8a2af030f29c1c9"
server_key_version: "encryptionkey@0"
I20260619 17:02:30.049520 9342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "8a44094d4434494f8d7aad772c830663"
format_stamp: "Formatted at 2026-06-19 17:02:30 on dist-test-slave-vnbj"
server_key: "e8d638d06fe3bc23d9e55d53feb2c4c6"
server_key_iv: "20bed768df5343c5f8a2af030f29c1c9"
server_key_version: "encryptionkey@0"
I20260619 17:02:30.050989 9342 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 17:02:30.051839 9357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:30.052115 9342 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:30.052222 9342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "8a44094d4434494f8d7aad772c830663"
format_stamp: "Formatted at 2026-06-19 17:02:30 on dist-test-slave-vnbj"
server_key: "e8d638d06fe3bc23d9e55d53feb2c4c6"
server_key_iv: "20bed768df5343c5f8a2af030f29c1c9"
server_key_version: "encryptionkey@0"
I20260619 17:02:30.052280 9342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:30.056082 9342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:30.058764 9342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:30.059017 9342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:30.063479 9342 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45767
I20260619 17:02:30.063555 9409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45767 every 8 connection(s)
I20260619 17:02:30.063922 9342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 17:02:30.065035 9410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:30.067655 9410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a44094d4434494f8d7aad772c830663: Bootstrap starting.
I20260619 17:02:30.068323 9410 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a44094d4434494f8d7aad772c830663: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:30.068658 9410 log.cc:824] T 00000000000000000000000000000000 P 8a44094d4434494f8d7aad772c830663: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:30.069476 9410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a44094d4434494f8d7aad772c830663: No bootstrap required, opened a new log
I20260619 17:02:30.071235 9410 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a44094d4434494f8d7aad772c830663 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a44094d4434494f8d7aad772c830663" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45767 } }
I20260619 17:02:30.071415 9410 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a44094d4434494f8d7aad772c830663 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:30.071465 9410 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a44094d4434494f8d7aad772c830663 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a44094d4434494f8d7aad772c830663, State: Initialized, Role: FOLLOWER
I20260619 17:02:30.071650 9410 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a44094d4434494f8d7aad772c830663 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a44094d4434494f8d7aad772c830663" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45767 } }
I20260619 17:02:30.071756 9410 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a44094d4434494f8d7aad772c830663 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 17:02:30.071799 9410 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a44094d4434494f8d7aad772c830663 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 17:02:30.071858 9410 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a44094d4434494f8d7aad772c830663 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:30.072453 9410 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a44094d4434494f8d7aad772c830663 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a44094d4434494f8d7aad772c830663" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45767 } }
I20260619 17:02:30.072582 9410 leader_election.cc:302] T 00000000000000000000000000000000 P 8a44094d4434494f8d7aad772c830663 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a44094d4434494f8d7aad772c830663; no voters:
I20260619 17:02:30.072820 9410 leader_election.cc:288] T 00000000000000000000000000000000 P 8a44094d4434494f8d7aad772c830663 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 17:02:30.072845 9413 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a44094d4434494f8d7aad772c830663 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:02:30.072849 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 9342
I20260619 17:02:30.072970 9413 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a44094d4434494f8d7aad772c830663 [term 1 LEADER]: Becoming Leader. State: Replica: 8a44094d4434494f8d7aad772c830663, State: Running, Role: LEADER
I20260619 17:02:30.072983 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 17:02:30.073056 9413 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a44094d4434494f8d7aad772c830663 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a44094d4434494f8d7aad772c830663" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45767 } }
I20260619 17:02:30.073134 9410 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a44094d4434494f8d7aad772c830663 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:02:30.073310 21406 external_mini_cluster.cc:1428] Setting key c2fc12fa45c99609f3cf7779d498eeec
I20260619 17:02:30.073393 9413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a44094d4434494f8d7aad772c830663 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a44094d4434494f8d7aad772c830663. Latest consensus state: current_term: 1 leader_uuid: "8a44094d4434494f8d7aad772c830663" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a44094d4434494f8d7aad772c830663" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45767 } } }
I20260619 17:02:30.073681 9415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a44094d4434494f8d7aad772c830663 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a44094d4434494f8d7aad772c830663" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a44094d4434494f8d7aad772c830663" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45767 } } }
I20260619 17:02:30.073971 9415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a44094d4434494f8d7aad772c830663 [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:30.073468 9413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a44094d4434494f8d7aad772c830663 [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:30.074324 9424 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 17:02:30.074992 9424 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 17:02:30.076238 9423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:30.074041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51761 (local address 127.0.0.1:45767)
0619 17:02:30.074465 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:30.074475 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:30.074512 (+ 37us) server_negotiation.cc:407] Connection header received
0619 17:02:30.074644 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:30.074745 (+ 101us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:30.074783 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:30.074997 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:30.075733 (+ 736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:30.075737 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:30.075737 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:30.075741 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:30.075789 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:30.075805 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:30.075882 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:30.076010 (+ 128us) server_negotiation.cc:299] Negotiation successful
0619 17:02:30.076051 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":88,"threads_started":1}
I20260619 17:02:30.077338 21406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 17:02:30.080078 9423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:30.078955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34229 (local address 127.0.0.1:45767)
0619 17:02:30.078998 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:30.079000 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:30.079160 (+ 160us) server_negotiation.cc:407] Connection header received
0619 17:02:30.079181 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:30.079182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:30.079186 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:30.079226 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:30.079842 (+ 616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:30.079843 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:30.079843 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:30.079846 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:30.079861 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:30.079865 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:30.079907 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:30.079968 (+ 61us) server_negotiation.cc:299] Negotiation successful
0619 17:02:30.079979 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260619 17:02:30.083741 9424 catalog_manager.cc:1347] Generated new cluster ID: 4f5aee3b087441e198abefb933487d50
I20260619 17:02:30.083829 9424 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 17:02:30.103462 9424 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 17:02:30.103741 9424 catalog_manager.cc:1504] Loading token signing keys...
I20260619 17:02:30.112074 9424 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a44094d4434494f8d7aad772c830663: Generated new TSK 0
I20260619 17:02:30.112381 9424 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 17:02:30.115862 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 9342
2026-06-19T17:02:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1357 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 17:02:30 dist-test-slave-vnbj krb5kdc[9441](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 17:02:30 dist-test-slave-vnbj krb5kdc[9441](info): set up 2 sockets
Jun 19 17:02:30 dist-test-slave-vnbj krb5kdc[9441](info): commencing operation
krb5kdc: starting...
W20260619 17:02:31.175999 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 17:02:31 dist-test-slave-vnbj krb5kdc[9441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T17:02:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T17:02:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 17:02:31.302517 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42965
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42965
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:02:31.386808 9454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:31.387099 9454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:31.387123 9454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:31.388288 9454 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:02:31.388329 9454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:31.388342 9454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:31.388355 9454 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:02:31.388368 9454 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:02:31.389856 9454 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42965
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:31.390115 9454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:31.390300 9454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:31.395789 9462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:31.395854 9454 server_base.cc:1034] running on GCE node
W20260619 17:02:31.395799 9460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:31.395818 9459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:31.396333 9454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:31.396668 9454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:31.397918 9454 hybrid_clock.cc:648] HybridClock initialized: now 1781888551397902 us; error 35 us; skew 500 ppm
Jun 19 17:02:31 dist-test-slave-vnbj krb5kdc[9441](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888551, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:31.400918 9454 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 17:02:31.401472 9454 webserver.cc:460] Webserver started at http://127.0.0.1:38969/ using document root <none> and password file <none>
I20260619 17:02:31.401732 9454 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:31.401805 9454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:31.401968 9454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:31.403252 9454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "f584f6e58fb447f381a49e27a6bd89c6"
format_stamp: "Formatted at 2026-06-19 17:02:31 on dist-test-slave-vnbj"
server_key: "3a530cdd176ff909697a1739b0a8f75f"
server_key_iv: "8432c0a96f9a78f5125b2f11ac06fc13"
server_key_version: "encryptionkey@0"
I20260619 17:02:31.403636 9454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "f584f6e58fb447f381a49e27a6bd89c6"
format_stamp: "Formatted at 2026-06-19 17:02:31 on dist-test-slave-vnbj"
server_key: "3a530cdd176ff909697a1739b0a8f75f"
server_key_iv: "8432c0a96f9a78f5125b2f11ac06fc13"
server_key_version: "encryptionkey@0"
I20260619 17:02:31.405113 9454 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 17:02:31.405949 9469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:31.406133 9454 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:31.406245 9454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "f584f6e58fb447f381a49e27a6bd89c6"
format_stamp: "Formatted at 2026-06-19 17:02:31 on dist-test-slave-vnbj"
server_key: "3a530cdd176ff909697a1739b0a8f75f"
server_key_iv: "8432c0a96f9a78f5125b2f11ac06fc13"
server_key_version: "encryptionkey@0"
I20260619 17:02:31.406325 9454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:31.418809 9454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:31.421422 9454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:31.421603 9454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:31.426048 9454 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42965
I20260619 17:02:31.426224 9521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42965 every 8 connection(s)
I20260619 17:02:31.426523 9454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 17:02:31.427587 9522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:31.428216 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 9454
I20260619 17:02:31.428359 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 17:02:31.428575 21406 external_mini_cluster.cc:1428] Setting key 107926f73d45d32343503d139a82dd75
I20260619 17:02:31.430836 9522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f584f6e58fb447f381a49e27a6bd89c6: Bootstrap starting.
I20260619 17:02:31.431618 9522 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f584f6e58fb447f381a49e27a6bd89c6: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:31.431958 9522 log.cc:824] T 00000000000000000000000000000000 P f584f6e58fb447f381a49e27a6bd89c6: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:31.432735 9522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f584f6e58fb447f381a49e27a6bd89c6: No bootstrap required, opened a new log
I20260619 17:02:31.434751 9522 raft_consensus.cc:348] T 00000000000000000000000000000000 P f584f6e58fb447f381a49e27a6bd89c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f584f6e58fb447f381a49e27a6bd89c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42965 } }
I20260619 17:02:31.434966 9522 raft_consensus.cc:374] T 00000000000000000000000000000000 P f584f6e58fb447f381a49e27a6bd89c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:31.434996 9522 raft_consensus.cc:729] T 00000000000000000000000000000000 P f584f6e58fb447f381a49e27a6bd89c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f584f6e58fb447f381a49e27a6bd89c6, State: Initialized, Role: FOLLOWER
I20260619 17:02:31.435141 9522 consensus_queue.cc:260] T 00000000000000000000000000000000 P f584f6e58fb447f381a49e27a6bd89c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f584f6e58fb447f381a49e27a6bd89c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42965 } }
I20260619 17:02:31.435205 9522 raft_consensus.cc:388] T 00000000000000000000000000000000 P f584f6e58fb447f381a49e27a6bd89c6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 17:02:31.435256 9522 raft_consensus.cc:482] T 00000000000000000000000000000000 P f584f6e58fb447f381a49e27a6bd89c6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 17:02:31.435308 9522 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f584f6e58fb447f381a49e27a6bd89c6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:31.435928 9522 raft_consensus.cc:504] T 00000000000000000000000000000000 P f584f6e58fb447f381a49e27a6bd89c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f584f6e58fb447f381a49e27a6bd89c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42965 } }
Jun 19 17:02:31 dist-test-slave-vnbj krb5kdc[9441](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 17:02:31.436053 9522 leader_election.cc:302] T 00000000000000000000000000000000 P f584f6e58fb447f381a49e27a6bd89c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f584f6e58fb447f381a49e27a6bd89c6; no voters:
I20260619 17:02:31.436246 9522 leader_election.cc:288] T 00000000000000000000000000000000 P f584f6e58fb447f381a49e27a6bd89c6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 17:02:31.436297 9527 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f584f6e58fb447f381a49e27a6bd89c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:02:31.436506 9527 raft_consensus.cc:686] T 00000000000000000000000000000000 P f584f6e58fb447f381a49e27a6bd89c6 [term 1 LEADER]: Becoming Leader. State: Replica: f584f6e58fb447f381a49e27a6bd89c6, State: Running, Role: LEADER
I20260619 17:02:31.436542 9522 sys_catalog.cc:564] T 00000000000000000000000000000000 P f584f6e58fb447f381a49e27a6bd89c6 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:02:31.436672 9527 consensus_queue.cc:237] T 00000000000000000000000000000000 P f584f6e58fb447f381a49e27a6bd89c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f584f6e58fb447f381a49e27a6bd89c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42965 } }
I20260619 17:02:31.437062 9529 sys_catalog.cc:455] T 00000000000000000000000000000000 P f584f6e58fb447f381a49e27a6bd89c6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f584f6e58fb447f381a49e27a6bd89c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f584f6e58fb447f381a49e27a6bd89c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42965 } } }
I20260619 17:02:31.437252 9529 sys_catalog.cc:458] T 00000000000000000000000000000000 P f584f6e58fb447f381a49e27a6bd89c6 [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:31.437050 9527 sys_catalog.cc:455] T 00000000000000000000000000000000 P f584f6e58fb447f381a49e27a6bd89c6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f584f6e58fb447f381a49e27a6bd89c6. Latest consensus state: current_term: 1 leader_uuid: "f584f6e58fb447f381a49e27a6bd89c6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f584f6e58fb447f381a49e27a6bd89c6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42965 } } }
I20260619 17:02:31.437551 9527 sys_catalog.cc:458] T 00000000000000000000000000000000 P f584f6e58fb447f381a49e27a6bd89c6 [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:31.437731 9536 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 17:02:31.438298 9536 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 17:02:31.440068 9536 catalog_manager.cc:1347] Generated new cluster ID: 7342a759aa574493858b957dae7f67de
I20260619 17:02:31.440126 9536 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 17:02:31.440994 9525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:31.429177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44863 (local address 127.0.0.1:42965)
0619 17:02:31.429392 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:31.429409 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:31.429445 (+ 36us) server_negotiation.cc:407] Connection header received
0619 17:02:31.430025 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:31.430051 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:31.430118 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:31.430277 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:31.430961 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:31.432357 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:31.433240 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:31.433703 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:31.433855 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:31.436600 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:31.436604 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:31.436605 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:31.436618 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:31.438813 (+ 2195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:31.439291 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:31.439293 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:31.439293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:31.439387 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:31.439612 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:31.439613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:31.439613 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:31.439969 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:31.440573 (+ 604us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:31.440640 (+ 67us) server_negotiation.cc:299] Negotiation successful
0619 17:02:31.440775 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":87,"threads_started":1}
I20260619 17:02:31.441777 21406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 17:02:31.455417 9525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:31.447347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52929 (local address 127.0.0.1:42965)
0619 17:02:31.447398 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:31.447400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:31.447588 (+ 188us) server_negotiation.cc:407] Connection header received
0619 17:02:31.447654 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:31.447655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:31.447668 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:31.447724 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:31.448490 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:31.449228 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:31.450348 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:31.450701 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:31.450768 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:31.452371 (+ 1603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:31.452373 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:31.452373 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:31.452381 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:31.454065 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:31.454528 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:31.454529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:31.454529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:31.454595 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:31.454810 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:31.454811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:31.454812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:31.454982 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:31.455076 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:31.455227 (+ 151us) server_negotiation.cc:299] Negotiation successful
0619 17:02:31.455317 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 17:02:31.460970 9536 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 17:02:31.461581 9536 catalog_manager.cc:1504] Loading token signing keys...
I20260619 17:02:31.464538 9536 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f584f6e58fb447f381a49e27a6bd89c6: Generated new TSK 0
I20260619 17:02:31.464752 9536 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 17:02:31.468775 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 9454
2026-06-19T17:02:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1352 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 17:02:31 dist-test-slave-vnbj krb5kdc[9553](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 17:02:31 dist-test-slave-vnbj krb5kdc[9553](info): set up 2 sockets
Jun 19 17:02:31 dist-test-slave-vnbj krb5kdc[9553](info): commencing operation
krb5kdc: starting...
W20260619 17:02:32.539194 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.052s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 17:02:32 dist-test-slave-vnbj krb5kdc[9553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888552, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T17:02:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T17:02:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 17:02:32.672498 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43187
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43187
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:02:32.757246 9566 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:32.757481 9566 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:32.757509 9566 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:32.758715 9566 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:02:32.758762 9566 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:32.758777 9566 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:32.758790 9566 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:02:32.758803 9566 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:02:32.760440 9566 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43187
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43187
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9566
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:32.760667 9566 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:32.760851 9566 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:32.766396 9574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:32.766431 9571 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:32.766433 9572 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:32.766638 9566 server_base.cc:1034] running on GCE node
I20260619 17:02:32.767033 9566 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:32.767369 9566 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:32.768591 9566 hybrid_clock.cc:648] HybridClock initialized: now 1781888552768562 us; error 39 us; skew 500 ppm
Jun 19 17:02:32 dist-test-slave-vnbj krb5kdc[9553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888552, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:32.777194 9566 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 17:02:32.777866 9566 webserver.cc:460] Webserver started at http://127.0.0.1:38119/ using document root <none> and password file <none>
I20260619 17:02:32.778144 9566 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:32.778220 9566 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:32.778393 9566 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:32.779654 9566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "fe8d557419d04be4820dddfad97d0a44"
format_stamp: "Formatted at 2026-06-19 17:02:32 on dist-test-slave-vnbj"
server_key: "49428fc0c96fa422e0dbc248c8f723a9"
server_key_iv: "ea0297dab4b77db80cb6a8201cd09ef2"
server_key_version: "encryptionkey@0"
I20260619 17:02:32.780081 9566 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "fe8d557419d04be4820dddfad97d0a44"
format_stamp: "Formatted at 2026-06-19 17:02:32 on dist-test-slave-vnbj"
server_key: "49428fc0c96fa422e0dbc248c8f723a9"
server_key_iv: "ea0297dab4b77db80cb6a8201cd09ef2"
server_key_version: "encryptionkey@0"
I20260619 17:02:32.781637 9566 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 17:02:32.782485 9581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:32.782725 9566 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:32.782820 9566 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "fe8d557419d04be4820dddfad97d0a44"
format_stamp: "Formatted at 2026-06-19 17:02:32 on dist-test-slave-vnbj"
server_key: "49428fc0c96fa422e0dbc248c8f723a9"
server_key_iv: "ea0297dab4b77db80cb6a8201cd09ef2"
server_key_version: "encryptionkey@0"
I20260619 17:02:32.782960 9566 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:32.824115 9566 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:32.826998 9566 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:32.827183 9566 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:32.831287 9566 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43187
I20260619 17:02:32.831645 9633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43187 every 8 connection(s)
I20260619 17:02:32.831691 9566 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 17:02:32.832707 9634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:32.835284 9634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe8d557419d04be4820dddfad97d0a44: Bootstrap starting.
I20260619 17:02:32.835917 9634 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fe8d557419d04be4820dddfad97d0a44: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:32.836208 9634 log.cc:824] T 00000000000000000000000000000000 P fe8d557419d04be4820dddfad97d0a44: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:32.837064 9634 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fe8d557419d04be4820dddfad97d0a44: No bootstrap required, opened a new log
I20260619 17:02:32.838694 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 9566
I20260619 17:02:32.838871 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 17:02:32.838795 9634 raft_consensus.cc:348] T 00000000000000000000000000000000 P fe8d557419d04be4820dddfad97d0a44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe8d557419d04be4820dddfad97d0a44" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43187 } }
I20260619 17:02:32.839062 9634 raft_consensus.cc:374] T 00000000000000000000000000000000 P fe8d557419d04be4820dddfad97d0a44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:32.839093 9634 raft_consensus.cc:729] T 00000000000000000000000000000000 P fe8d557419d04be4820dddfad97d0a44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe8d557419d04be4820dddfad97d0a44, State: Initialized, Role: FOLLOWER
I20260619 17:02:32.839114 21406 external_mini_cluster.cc:1428] Setting key 6368a5eae3458e08caf1e862e2dd0983
I20260619 17:02:32.839278 9634 consensus_queue.cc:260] T 00000000000000000000000000000000 P fe8d557419d04be4820dddfad97d0a44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe8d557419d04be4820dddfad97d0a44" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43187 } }
I20260619 17:02:32.839399 9634 raft_consensus.cc:388] T 00000000000000000000000000000000 P fe8d557419d04be4820dddfad97d0a44 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 17:02:32.839437 9634 raft_consensus.cc:482] T 00000000000000000000000000000000 P fe8d557419d04be4820dddfad97d0a44 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 17:02:32.839493 9634 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fe8d557419d04be4820dddfad97d0a44 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:32.840914 9634 raft_consensus.cc:504] T 00000000000000000000000000000000 P fe8d557419d04be4820dddfad97d0a44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe8d557419d04be4820dddfad97d0a44" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43187 } }
I20260619 17:02:32.841027 9634 leader_election.cc:302] T 00000000000000000000000000000000 P fe8d557419d04be4820dddfad97d0a44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fe8d557419d04be4820dddfad97d0a44; no voters:
I20260619 17:02:32.841185 9634 leader_election.cc:288] T 00000000000000000000000000000000 P fe8d557419d04be4820dddfad97d0a44 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 17:02:32.841305 9639 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fe8d557419d04be4820dddfad97d0a44 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:02:32.841454 9634 sys_catalog.cc:564] T 00000000000000000000000000000000 P fe8d557419d04be4820dddfad97d0a44 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:02:32.841492 9639 raft_consensus.cc:686] T 00000000000000000000000000000000 P fe8d557419d04be4820dddfad97d0a44 [term 1 LEADER]: Becoming Leader. State: Replica: fe8d557419d04be4820dddfad97d0a44, State: Running, Role: LEADER
I20260619 17:02:32.841629 9639 consensus_queue.cc:237] T 00000000000000000000000000000000 P fe8d557419d04be4820dddfad97d0a44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe8d557419d04be4820dddfad97d0a44" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43187 } }
I20260619 17:02:32.842034 9641 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe8d557419d04be4820dddfad97d0a44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fe8d557419d04be4820dddfad97d0a44. Latest consensus state: current_term: 1 leader_uuid: "fe8d557419d04be4820dddfad97d0a44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe8d557419d04be4820dddfad97d0a44" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43187 } } }
I20260619 17:02:32.842299 9641 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe8d557419d04be4820dddfad97d0a44 [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:32.842204 9640 sys_catalog.cc:455] T 00000000000000000000000000000000 P fe8d557419d04be4820dddfad97d0a44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fe8d557419d04be4820dddfad97d0a44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fe8d557419d04be4820dddfad97d0a44" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43187 } } }
I20260619 17:02:32.842614 9640 sys_catalog.cc:458] T 00000000000000000000000000000000 P fe8d557419d04be4820dddfad97d0a44 [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:32.842782 9645 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 17:02:32.843461 9645 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 17:02:32.845490 9645 catalog_manager.cc:1347] Generated new cluster ID: c6e746719ea6469cadea0550f31f9875
I20260619 17:02:32.845541 9638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:32.839640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51127 (local address 127.0.0.1:43187)
0619 17:02:32.839888 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:32.839897 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:32.839934 (+ 37us) server_negotiation.cc:407] Connection header received
0619 17:02:32.840076 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:32.840087 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:32.840171 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:32.840368 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:32.840671 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:32.842016 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:32.843303 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:32.843926 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:32.844048 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:32.844809 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:32.844813 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:32.844813 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:32.845021 (+ 208us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:32.845061 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:32.845077 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:32.845146 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:32.845246 (+ 100us) server_negotiation.cc:299] Negotiation successful
0619 17:02:32.845314 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":133,"threads_started":1}
I20260619 17:02:32.845544 9645 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 17:02:32.846472 21406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 17:02:32.854285 9638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:32.847931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38337 (local address 127.0.0.1:43187)
0619 17:02:32.847971 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:32.847973 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:32.851161 (+ 3188us) server_negotiation.cc:407] Connection header received
0619 17:02:32.851249 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:32.851251 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:32.851264 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:32.851318 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:32.851490 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:32.852090 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:32.852902 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:32.853134 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:32.853219 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:32.853994 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:32.853996 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:32.853996 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:32.854002 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:32.854019 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:32.854023 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:32.854075 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:32.854143 (+ 68us) server_negotiation.cc:299] Negotiation successful
0619 17:02:32.854174 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 17:02:32.869305 9645 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 17:02:32.870113 9645 catalog_manager.cc:1504] Loading token signing keys...
I20260619 17:02:32.873477 9645 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fe8d557419d04be4820dddfad97d0a44: Generated new TSK 0
I20260619 17:02:32.873699 9645 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 17:02:32.881603 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 9566
2026-06-19T17:02:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1412 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 17:02:32 dist-test-slave-vnbj krb5kdc[9665](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 17:02:32 dist-test-slave-vnbj krb5kdc[9665](info): set up 2 sockets
Jun 19 17:02:32 dist-test-slave-vnbj krb5kdc[9665](info): commencing operation
krb5kdc: starting...
W20260619 17:02:33.947271 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.047s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 17:02:34 dist-test-slave-vnbj krb5kdc[9665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888554, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T17:02:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T17:02:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 17:02:34.082994 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39887
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39887
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:02:34.169605 9678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:34.169915 9678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:34.169963 9678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:34.171259 9678 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:02:34.171357 9678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:34.171386 9678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:34.171412 9678 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:02:34.171446 9678 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:02:34.173198 9678 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9678
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:34.173539 9678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:34.173923 9678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:34.179553 9686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:34.179561 9684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:34.179775 9683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:34.179677 9678 server_base.cc:1034] running on GCE node
I20260619 17:02:34.180099 9678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:34.180408 9678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:34.181566 9678 hybrid_clock.cc:648] HybridClock initialized: now 1781888554181548 us; error 29 us; skew 500 ppm
Jun 19 17:02:34 dist-test-slave-vnbj krb5kdc[9665](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888554, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:34.184374 9678 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 17:02:34.184906 9678 webserver.cc:460] Webserver started at http://127.0.0.1:43981/ using document root <none> and password file <none>
I20260619 17:02:34.185104 9678 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:34.185148 9678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:34.185241 9678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:34.186343 9678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "75bae9dde2844207bcc5e25deca3a56f"
format_stamp: "Formatted at 2026-06-19 17:02:34 on dist-test-slave-vnbj"
server_key: "2d1624bc2941211576d56ddde2555090"
server_key_iv: "cad075f2e4940d0477c72eb3b59399ca"
server_key_version: "encryptionkey@0"
I20260619 17:02:34.186677 9678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "75bae9dde2844207bcc5e25deca3a56f"
format_stamp: "Formatted at 2026-06-19 17:02:34 on dist-test-slave-vnbj"
server_key: "2d1624bc2941211576d56ddde2555090"
server_key_iv: "cad075f2e4940d0477c72eb3b59399ca"
server_key_version: "encryptionkey@0"
I20260619 17:02:34.188438 9678 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260619 17:02:34.189457 9693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:34.189674 9678 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:34.189776 9678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "75bae9dde2844207bcc5e25deca3a56f"
format_stamp: "Formatted at 2026-06-19 17:02:34 on dist-test-slave-vnbj"
server_key: "2d1624bc2941211576d56ddde2555090"
server_key_iv: "cad075f2e4940d0477c72eb3b59399ca"
server_key_version: "encryptionkey@0"
I20260619 17:02:34.189833 9678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:34.193276 9678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:34.195662 9678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:34.195772 9678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:34.199839 9678 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39887
I20260619 17:02:34.199920 9745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39887 every 8 connection(s)
I20260619 17:02:34.200261 9678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 17:02:34.201329 9746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:34.203877 9746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75bae9dde2844207bcc5e25deca3a56f: Bootstrap starting.
I20260619 17:02:34.204514 9746 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75bae9dde2844207bcc5e25deca3a56f: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:34.204823 9746 log.cc:824] T 00000000000000000000000000000000 P 75bae9dde2844207bcc5e25deca3a56f: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:34.205597 9746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75bae9dde2844207bcc5e25deca3a56f: No bootstrap required, opened a new log
I20260619 17:02:34.207417 9746 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75bae9dde2844207bcc5e25deca3a56f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75bae9dde2844207bcc5e25deca3a56f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39887 } }
I20260619 17:02:34.207587 9746 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75bae9dde2844207bcc5e25deca3a56f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:34.207638 9746 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75bae9dde2844207bcc5e25deca3a56f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75bae9dde2844207bcc5e25deca3a56f, State: Initialized, Role: FOLLOWER
I20260619 17:02:34.207832 9746 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75bae9dde2844207bcc5e25deca3a56f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75bae9dde2844207bcc5e25deca3a56f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39887 } }
I20260619 17:02:34.207937 9746 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75bae9dde2844207bcc5e25deca3a56f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 17:02:34.207981 9746 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75bae9dde2844207bcc5e25deca3a56f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 17:02:34.208031 9746 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75bae9dde2844207bcc5e25deca3a56f [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:34.208506 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 9678
I20260619 17:02:34.208630 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 17:02:34.208623 9746 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75bae9dde2844207bcc5e25deca3a56f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75bae9dde2844207bcc5e25deca3a56f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39887 } }
I20260619 17:02:34.208725 9746 leader_election.cc:302] T 00000000000000000000000000000000 P 75bae9dde2844207bcc5e25deca3a56f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75bae9dde2844207bcc5e25deca3a56f; no voters:
I20260619 17:02:34.208843 21406 external_mini_cluster.cc:1428] Setting key 073c0e96036b0b3f5cff47f7c87f7aba
I20260619 17:02:34.208935 9746 leader_election.cc:288] T 00000000000000000000000000000000 P 75bae9dde2844207bcc5e25deca3a56f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 17:02:34.209009 9749 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75bae9dde2844207bcc5e25deca3a56f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:02:34.209195 9749 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75bae9dde2844207bcc5e25deca3a56f [term 1 LEADER]: Becoming Leader. State: Replica: 75bae9dde2844207bcc5e25deca3a56f, State: Running, Role: LEADER
I20260619 17:02:34.209213 9746 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75bae9dde2844207bcc5e25deca3a56f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:02:34.209297 9749 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75bae9dde2844207bcc5e25deca3a56f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75bae9dde2844207bcc5e25deca3a56f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39887 } }
I20260619 17:02:34.209707 9750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75bae9dde2844207bcc5e25deca3a56f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "75bae9dde2844207bcc5e25deca3a56f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75bae9dde2844207bcc5e25deca3a56f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39887 } } }
I20260619 17:02:34.209794 9750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75bae9dde2844207bcc5e25deca3a56f [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:34.210445 9751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75bae9dde2844207bcc5e25deca3a56f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75bae9dde2844207bcc5e25deca3a56f. Latest consensus state: current_term: 1 leader_uuid: "75bae9dde2844207bcc5e25deca3a56f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75bae9dde2844207bcc5e25deca3a56f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39887 } } }
I20260619 17:02:34.210609 9751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75bae9dde2844207bcc5e25deca3a56f [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:34.210521 9757 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 17:02:34.211241 9757 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 17:02:34.212427 9761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:34.210020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55449 (local address 127.0.0.1:39887)
0619 17:02:34.210748 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:34.210759 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:34.210793 (+ 34us) server_negotiation.cc:407] Connection header received
0619 17:02:34.210968 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:34.210974 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:34.210999 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:34.211155 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:34.211912 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:34.211915 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:34.211916 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:34.211920 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:34.211965 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:34.211981 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:34.212036 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:34.212195 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 17:02:34.212239 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":634,"threads_started":1}
I20260619 17:02:34.213493 21406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 17:02:34.216387 9761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:34.215208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33805 (local address 127.0.0.1:39887)
0619 17:02:34.215268 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:34.215271 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:34.215443 (+ 172us) server_negotiation.cc:407] Connection header received
0619 17:02:34.215518 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:34.215519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:34.215524 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:34.215571 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:34.216176 (+ 605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:34.216177 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:34.216177 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:34.216180 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:34.216193 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:34.216197 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:34.216238 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:34.216329 (+ 91us) server_negotiation.cc:299] Negotiation successful
0619 17:02:34.216340 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 17:02:34.219403 9757 catalog_manager.cc:1347] Generated new cluster ID: 58fd1b5641da4285bfec65dd576e638b
I20260619 17:02:34.219476 9757 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 17:02:34.226444 9757 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 17:02:34.226673 9757 catalog_manager.cc:1504] Loading token signing keys...
I20260619 17:02:34.232398 9757 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 75bae9dde2844207bcc5e25deca3a56f: Generated new TSK 0
I20260619 17:02:34.232677 9757 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 17:02:34.237813 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 9678
2026-06-19T17:02:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1356 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 17:02:34 dist-test-slave-vnbj krb5kdc[9777](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 17:02:34 dist-test-slave-vnbj krb5kdc[9777](info): set up 2 sockets
Jun 19 17:02:34 dist-test-slave-vnbj krb5kdc[9777](info): commencing operation
krb5kdc: starting...
W20260619 17:02:35.328328 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 17:02:35 dist-test-slave-vnbj krb5kdc[9777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T17:02:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T17:02:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 17:02:35.462747 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42983
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38229
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42983
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:02:35.560269 9790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:35.560555 9790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:35.560587 9790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:35.562409 9790 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:02:35.562506 9790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:35.562525 9790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:35.562541 9790 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:02:35.562554 9790 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:02:35.564935 9790 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38229
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42983
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:35.565328 9790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:35.565565 9790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:35.571619 9798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:35.571842 9790 server_base.cc:1034] running on GCE node
W20260619 17:02:35.571780 9795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:35.571643 9796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:35.572271 9790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:35.572618 9790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:35.573801 9790 hybrid_clock.cc:648] HybridClock initialized: now 1781888555573778 us; error 49 us; skew 500 ppm
Jun 19 17:02:35 dist-test-slave-vnbj krb5kdc[9777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888555, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:35.576778 9790 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 17:02:35.577389 9790 webserver.cc:460] Webserver started at http://127.0.0.1:41787/ using document root <none> and password file <none>
I20260619 17:02:35.577642 9790 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:35.577715 9790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:35.577867 9790 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:35.579123 9790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "443862b8fa2843019b7a3a6e5495f4ed"
format_stamp: "Formatted at 2026-06-19 17:02:35 on dist-test-slave-vnbj"
server_key: "2b60281e33dae8b9feb1b894551aff00"
server_key_iv: "597d2e9ff70abc524564522105f9df0c"
server_key_version: "encryptionkey@0"
I20260619 17:02:35.579496 9790 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "443862b8fa2843019b7a3a6e5495f4ed"
format_stamp: "Formatted at 2026-06-19 17:02:35 on dist-test-slave-vnbj"
server_key: "2b60281e33dae8b9feb1b894551aff00"
server_key_iv: "597d2e9ff70abc524564522105f9df0c"
server_key_version: "encryptionkey@0"
I20260619 17:02:35.581023 9790 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260619 17:02:35.581858 9805 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:35.582345 9790 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:35.582620 9790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "443862b8fa2843019b7a3a6e5495f4ed"
format_stamp: "Formatted at 2026-06-19 17:02:35 on dist-test-slave-vnbj"
server_key: "2b60281e33dae8b9feb1b894551aff00"
server_key_iv: "597d2e9ff70abc524564522105f9df0c"
server_key_version: "encryptionkey@0"
I20260619 17:02:35.582727 9790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:35.601631 9790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:35.604321 9790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:35.604521 9790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:35.609228 9790 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42983
I20260619 17:02:35.609297 9857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42983 every 8 connection(s)
I20260619 17:02:35.609740 9790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 17:02:35.610673 9858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:35.613313 9858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 443862b8fa2843019b7a3a6e5495f4ed: Bootstrap starting.
I20260619 17:02:35.613893 9858 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 443862b8fa2843019b7a3a6e5495f4ed: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:35.614144 9858 log.cc:824] T 00000000000000000000000000000000 P 443862b8fa2843019b7a3a6e5495f4ed: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:35.615020 9858 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 443862b8fa2843019b7a3a6e5495f4ed: No bootstrap required, opened a new log
I20260619 17:02:35.616674 9858 raft_consensus.cc:348] T 00000000000000000000000000000000 P 443862b8fa2843019b7a3a6e5495f4ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "443862b8fa2843019b7a3a6e5495f4ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42983 } }
I20260619 17:02:35.616850 9858 raft_consensus.cc:374] T 00000000000000000000000000000000 P 443862b8fa2843019b7a3a6e5495f4ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:35.616876 9858 raft_consensus.cc:729] T 00000000000000000000000000000000 P 443862b8fa2843019b7a3a6e5495f4ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 443862b8fa2843019b7a3a6e5495f4ed, State: Initialized, Role: FOLLOWER
I20260619 17:02:35.617039 9858 consensus_queue.cc:260] T 00000000000000000000000000000000 P 443862b8fa2843019b7a3a6e5495f4ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "443862b8fa2843019b7a3a6e5495f4ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42983 } }
I20260619 17:02:35.617126 9858 raft_consensus.cc:388] T 00000000000000000000000000000000 P 443862b8fa2843019b7a3a6e5495f4ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 17:02:35.617151 9858 raft_consensus.cc:482] T 00000000000000000000000000000000 P 443862b8fa2843019b7a3a6e5495f4ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 17:02:35.617182 9858 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 443862b8fa2843019b7a3a6e5495f4ed [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:35.617769 9858 raft_consensus.cc:504] T 00000000000000000000000000000000 P 443862b8fa2843019b7a3a6e5495f4ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "443862b8fa2843019b7a3a6e5495f4ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42983 } }
I20260619 17:02:35.617872 9858 leader_election.cc:302] T 00000000000000000000000000000000 P 443862b8fa2843019b7a3a6e5495f4ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 443862b8fa2843019b7a3a6e5495f4ed; no voters:
I20260619 17:02:35.618054 9858 leader_election.cc:288] T 00000000000000000000000000000000 P 443862b8fa2843019b7a3a6e5495f4ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 17:02:35.618084 9861 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 443862b8fa2843019b7a3a6e5495f4ed [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:02:35.618223 9861 raft_consensus.cc:686] T 00000000000000000000000000000000 P 443862b8fa2843019b7a3a6e5495f4ed [term 1 LEADER]: Becoming Leader. State: Replica: 443862b8fa2843019b7a3a6e5495f4ed, State: Running, Role: LEADER
I20260619 17:02:35.618350 9858 sys_catalog.cc:564] T 00000000000000000000000000000000 P 443862b8fa2843019b7a3a6e5495f4ed [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:02:35.618387 9861 consensus_queue.cc:237] T 00000000000000000000000000000000 P 443862b8fa2843019b7a3a6e5495f4ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "443862b8fa2843019b7a3a6e5495f4ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42983 } }
I20260619 17:02:35.619159 9862 sys_catalog.cc:455] T 00000000000000000000000000000000 P 443862b8fa2843019b7a3a6e5495f4ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 443862b8fa2843019b7a3a6e5495f4ed. Latest consensus state: current_term: 1 leader_uuid: "443862b8fa2843019b7a3a6e5495f4ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "443862b8fa2843019b7a3a6e5495f4ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42983 } } }
I20260619 17:02:35.619277 9862 sys_catalog.cc:458] T 00000000000000000000000000000000 P 443862b8fa2843019b7a3a6e5495f4ed [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:35.619453 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 9790
I20260619 17:02:35.619591 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 17:02:35.619621 9870 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 17:02:35.619710 9863 sys_catalog.cc:455] T 00000000000000000000000000000000 P 443862b8fa2843019b7a3a6e5495f4ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "443862b8fa2843019b7a3a6e5495f4ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "443862b8fa2843019b7a3a6e5495f4ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42983 } } }
I20260619 17:02:35.619772 9863 sys_catalog.cc:458] T 00000000000000000000000000000000 P 443862b8fa2843019b7a3a6e5495f4ed [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:35.619837 21406 external_mini_cluster.cc:1428] Setting key 014a023419f0c293d49b92be7f30d52a
I20260619 17:02:35.620322 9870 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 17:02:35.622225 9870 catalog_manager.cc:1347] Generated new cluster ID: 8a76e1ce9a634006a096964d1ec5c464
I20260619 17:02:35.622289 9870 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 17:02:35 dist-test-slave-vnbj krb5kdc[9777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888555, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 17:02:35.646092 9880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:35.620363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.139:53331 (local address 127.0.0.1:42983)
0619 17:02:35.631038 (+ 10675us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:35.631049 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:35.631096 (+ 47us) server_negotiation.cc:407] Connection header received
0619 17:02:35.631306 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:35.631319 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:35.631402 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:35.631618 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:35.632771 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:35.634443 (+ 1672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:35.635891 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:35.636669 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:35.636798 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:35.641108 (+ 4310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:35.641116 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:35.641117 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:35.641136 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:35.643901 (+ 2765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:35.644517 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:35.644523 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:35.644524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:35.644689 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:35.644986 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:35.644987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:35.644988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:35.645381 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:35.645592 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:35.645699 (+ 107us) server_negotiation.cc:299] Negotiation successful
0619 17:02:35.645820 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10601,"thread_start_us":83,"threads_started":1}
I20260619 17:02:35.647298 21406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 17:02:35.652860 9870 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 17:02:35.653626 9870 catalog_manager.cc:1504] Loading token signing keys...
I20260619 17:02:35.657754 9880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:35.649044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.139:55541 (local address 127.0.0.1:42983)
0619 17:02:35.649121 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:35.649123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:35.649182 (+ 59us) server_negotiation.cc:407] Connection header received
0619 17:02:35.649278 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:35.649279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:35.649291 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:35.649404 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:35.650163 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:35.651343 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:35.652395 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:35.652699 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:35.652787 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:35.654409 (+ 1622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:35.654410 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:35.654411 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:35.654419 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:35.656234 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:35.656790 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:35.656792 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:35.656792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:35.656859 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:35.657132 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:35.657133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:35.657133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:35.657280 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:35.657469 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:35.657570 (+ 101us) server_negotiation.cc:299] Negotiation successful
0619 17:02:35.657639 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 17:02:35.658529 9870 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 443862b8fa2843019b7a3a6e5495f4ed: Generated new TSK 0
I20260619 17:02:35.658740 9870 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 17:02:35.661845 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 9790
W20260619 17:02:35.666175 9775 connection.cc:535] client connection to 127.0.0.1:42983 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42983): Connection reset by peer (error 104)
2026-06-19T17:02:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1423 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 17:02:35 dist-test-slave-vnbj krb5kdc[9889](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 17:02:35 dist-test-slave-vnbj krb5kdc[9889](info): set up 2 sockets
Jun 19 17:02:35 dist-test-slave-vnbj krb5kdc[9889](info): commencing operation
krb5kdc: starting...
W20260619 17:02:36.799319 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 17:02:36 dist-test-slave-vnbj krb5kdc[9889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T17:02:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T17:02:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 17:02:36.939603 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36231
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36231
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:02:37.034017 9902 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:37.034264 9902 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:37.034291 9902 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:37.035665 9902 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:02:37.035725 9902 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:37.035740 9902 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:37.035753 9902 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:02:37.035765 9902 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:02:37.037384 9902 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36231
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9902
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:37.037609 9902 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:37.037791 9902 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:37.043165 9908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:37.043388 9902 server_base.cc:1034] running on GCE node
W20260619 17:02:37.043200 9910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:37.043198 9907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:37.043735 9902 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:37.044055 9902 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:37.045275 9902 hybrid_clock.cc:648] HybridClock initialized: now 1781888557045216 us; error 65 us; skew 500 ppm
Jun 19 17:02:37 dist-test-slave-vnbj krb5kdc[9889](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888557, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:37.048156 9902 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 17:02:37.048710 9902 webserver.cc:460] Webserver started at http://127.0.0.1:33733/ using document root <none> and password file <none>
I20260619 17:02:37.048928 9902 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:37.048974 9902 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:37.049075 9902 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:37.050257 9902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "57ac8135d6964954a12b73b504c615fc"
format_stamp: "Formatted at 2026-06-19 17:02:37 on dist-test-slave-vnbj"
server_key: "773da584b0aa24484aea9591e5583a55"
server_key_iv: "5d52273d7fa4089d9b255403ed1cc96c"
server_key_version: "encryptionkey@0"
I20260619 17:02:37.050626 9902 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "57ac8135d6964954a12b73b504c615fc"
format_stamp: "Formatted at 2026-06-19 17:02:37 on dist-test-slave-vnbj"
server_key: "773da584b0aa24484aea9591e5583a55"
server_key_iv: "5d52273d7fa4089d9b255403ed1cc96c"
server_key_version: "encryptionkey@0"
I20260619 17:02:37.052361 9902 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260619 17:02:37.053408 9917 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:37.053622 9902 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:37.053727 9902 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "57ac8135d6964954a12b73b504c615fc"
format_stamp: "Formatted at 2026-06-19 17:02:37 on dist-test-slave-vnbj"
server_key: "773da584b0aa24484aea9591e5583a55"
server_key_iv: "5d52273d7fa4089d9b255403ed1cc96c"
server_key_version: "encryptionkey@0"
I20260619 17:02:37.053787 9902 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:37.072120 9902 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:37.074966 9902 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:37.075106 9902 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:37.079231 9902 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36231
I20260619 17:02:37.079373 9969 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36231 every 8 connection(s)
I20260619 17:02:37.079704 9902 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 17:02:37.080708 9970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:37.083506 9970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57ac8135d6964954a12b73b504c615fc: Bootstrap starting.
I20260619 17:02:37.084167 9970 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57ac8135d6964954a12b73b504c615fc: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:37.084486 9970 log.cc:824] T 00000000000000000000000000000000 P 57ac8135d6964954a12b73b504c615fc: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:37.085405 9970 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57ac8135d6964954a12b73b504c615fc: No bootstrap required, opened a new log
I20260619 17:02:37.086903 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 9902
I20260619 17:02:37.087036 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 17:02:37.087244 21406 external_mini_cluster.cc:1428] Setting key 5d178fae9a800e6260c0bfbbcf72107f
I20260619 17:02:37.087724 9970 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57ac8135d6964954a12b73b504c615fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57ac8135d6964954a12b73b504c615fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36231 } }
I20260619 17:02:37.087960 9970 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57ac8135d6964954a12b73b504c615fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:37.088003 9970 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57ac8135d6964954a12b73b504c615fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57ac8135d6964954a12b73b504c615fc, State: Initialized, Role: FOLLOWER
I20260619 17:02:37.088153 9970 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57ac8135d6964954a12b73b504c615fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57ac8135d6964954a12b73b504c615fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36231 } }
I20260619 17:02:37.088238 9970 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57ac8135d6964954a12b73b504c615fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 17:02:37.088263 9970 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57ac8135d6964954a12b73b504c615fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 17:02:37.088331 9970 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57ac8135d6964954a12b73b504c615fc [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:37.089257 9970 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57ac8135d6964954a12b73b504c615fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57ac8135d6964954a12b73b504c615fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36231 } }
I20260619 17:02:37.089362 9970 leader_election.cc:302] T 00000000000000000000000000000000 P 57ac8135d6964954a12b73b504c615fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57ac8135d6964954a12b73b504c615fc; no voters:
I20260619 17:02:37.089620 9970 leader_election.cc:288] T 00000000000000000000000000000000 P 57ac8135d6964954a12b73b504c615fc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 17:02:37.089650 9975 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57ac8135d6964954a12b73b504c615fc [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:02:37.089771 9975 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57ac8135d6964954a12b73b504c615fc [term 1 LEADER]: Becoming Leader. State: Replica: 57ac8135d6964954a12b73b504c615fc, State: Running, Role: LEADER
I20260619 17:02:37.089991 9975 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57ac8135d6964954a12b73b504c615fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57ac8135d6964954a12b73b504c615fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36231 } }
I20260619 17:02:37.090046 9970 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57ac8135d6964954a12b73b504c615fc [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:02:37.090632 9975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57ac8135d6964954a12b73b504c615fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57ac8135d6964954a12b73b504c615fc. Latest consensus state: current_term: 1 leader_uuid: "57ac8135d6964954a12b73b504c615fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57ac8135d6964954a12b73b504c615fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36231 } } }
I20260619 17:02:37.090763 9975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57ac8135d6964954a12b73b504c615fc [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:37.091253 9984 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 17:02:37.091758 9976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57ac8135d6964954a12b73b504c615fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57ac8135d6964954a12b73b504c615fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57ac8135d6964954a12b73b504c615fc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36231 } } }
I20260619 17:02:37.091859 9976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57ac8135d6964954a12b73b504c615fc [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:37.091882 9984 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 17:02:37.094103 9984 catalog_manager.cc:1347] Generated new cluster ID: 7624203d3a694074bf181ef30c63a0fd
I20260619 17:02:37.094166 9984 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 17:02:37.094501 9974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:37.087869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.139:38563 (local address 127.0.0.1:36231)
0619 17:02:37.088119 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:37.088130 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:37.088169 (+ 39us) server_negotiation.cc:407] Connection header received
0619 17:02:37.088342 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:37.088349 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:37.088422 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:37.088584 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:37.088976 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:37.090720 (+ 1744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:37.092081 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:37.092687 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:37.092804 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:37.093643 (+ 839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:37.093648 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:37.093648 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:37.093886 (+ 238us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:37.093933 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:37.093950 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:37.094202 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:37.094247 (+ 45us) server_negotiation.cc:299] Negotiation successful
0619 17:02:37.094276 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":84,"threads_started":1}
I20260619 17:02:37.095610 21406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 17:02:37.103509 9974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:37.099503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.139:51643 (local address 127.0.0.1:36231)
0619 17:02:37.099573 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:37.099575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:37.099816 (+ 241us) server_negotiation.cc:407] Connection header received
0619 17:02:37.099858 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:37.099859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:37.099871 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:37.099935 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:37.100095 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:37.100889 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:37.101934 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:37.102268 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:37.102349 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:37.103206 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:37.103208 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:37.103208 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:37.103214 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:37.103241 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:37.103246 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:37.103315 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:37.103389 (+ 74us) server_negotiation.cc:299] Negotiation successful
0619 17:02:37.103405 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 17:02:37.109637 9984 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 17:02:37.110416 9984 catalog_manager.cc:1504] Loading token signing keys...
I20260619 17:02:37.123172 9984 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57ac8135d6964954a12b73b504c615fc: Generated new TSK 0
I20260619 17:02:37.123441 9984 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 17:02:37.129477 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 9902
W20260619 17:02:37.133805 9887 connection.cc:535] client connection to 127.0.0.1:36231 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36231): Connection reset by peer (error 104)
2026-06-19T17:02:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1467 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 17:02:37 dist-test-slave-vnbj krb5kdc[10001](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 17:02:37 dist-test-slave-vnbj krb5kdc[10001](info): set up 2 sockets
Jun 19 17:02:37 dist-test-slave-vnbj krb5kdc[10001](info): commencing operation
krb5kdc: starting...
W20260619 17:02:38.191215 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 17:02:38 dist-test-slave-vnbj krb5kdc[10001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T17:02:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T17:02:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 17:02:38.326455 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37353
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37353
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:02:38.414134 10014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:38.414434 10014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:38.414479 10014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:38.415874 10014 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:02:38.415971 10014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:38.415997 10014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:38.416054 10014 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:02:38.416081 10014 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:02:38.417780 10014 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37353
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37353
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10014
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:38.418133 10014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:38.418392 10014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:38.423822 10022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:38.423834 10020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:38.423929 10019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:38.424206 10014 server_base.cc:1034] running on GCE node
I20260619 17:02:38.424491 10014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:38.424815 10014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:38.426020 10014 hybrid_clock.cc:648] HybridClock initialized: now 1781888558426003 us; error 31 us; skew 500 ppm
Jun 19 17:02:38 dist-test-slave-vnbj krb5kdc[10001](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888558, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:38.435045 10014 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 17:02:38.435698 10014 webserver.cc:460] Webserver started at http://127.0.0.1:38551/ using document root <none> and password file <none>
I20260619 17:02:38.435971 10014 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:38.436048 10014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:38.436210 10014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:38.437427 10014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "95d50018b87d4e81b431ae066465c74a"
format_stamp: "Formatted at 2026-06-19 17:02:38 on dist-test-slave-vnbj"
server_key: "786eaf7a2f075d112d31c3e42c0afd35"
server_key_iv: "3ec3d7311232ce57d45ebf8df1ff5f67"
server_key_version: "encryptionkey@0"
I20260619 17:02:38.437844 10014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "95d50018b87d4e81b431ae066465c74a"
format_stamp: "Formatted at 2026-06-19 17:02:38 on dist-test-slave-vnbj"
server_key: "786eaf7a2f075d112d31c3e42c0afd35"
server_key_iv: "3ec3d7311232ce57d45ebf8df1ff5f67"
server_key_version: "encryptionkey@0"
I20260619 17:02:38.439517 10014 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260619 17:02:38.440539 10029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:38.440905 10014 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:38.441048 10014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "95d50018b87d4e81b431ae066465c74a"
format_stamp: "Formatted at 2026-06-19 17:02:38 on dist-test-slave-vnbj"
server_key: "786eaf7a2f075d112d31c3e42c0afd35"
server_key_iv: "3ec3d7311232ce57d45ebf8df1ff5f67"
server_key_version: "encryptionkey@0"
I20260619 17:02:38.441135 10014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:38.444938 10014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:38.447662 10014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:38.447862 10014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:38.452777 10014 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37353
I20260619 17:02:38.452828 10081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37353 every 8 connection(s)
I20260619 17:02:38.453214 10014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 17:02:38.454325 10082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:38.456977 10082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95d50018b87d4e81b431ae066465c74a: Bootstrap starting.
I20260619 17:02:38.457609 10082 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95d50018b87d4e81b431ae066465c74a: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:38.457908 10082 log.cc:824] T 00000000000000000000000000000000 P 95d50018b87d4e81b431ae066465c74a: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:38.458760 10082 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95d50018b87d4e81b431ae066465c74a: No bootstrap required, opened a new log
I20260619 17:02:38.460503 10082 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95d50018b87d4e81b431ae066465c74a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95d50018b87d4e81b431ae066465c74a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37353 } }
I20260619 17:02:38.460659 10082 raft_consensus.cc:374] T 00000000000000000000000000000000 P 95d50018b87d4e81b431ae066465c74a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:38.460682 10082 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95d50018b87d4e81b431ae066465c74a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95d50018b87d4e81b431ae066465c74a, State: Initialized, Role: FOLLOWER
I20260619 17:02:38.460848 10082 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95d50018b87d4e81b431ae066465c74a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95d50018b87d4e81b431ae066465c74a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37353 } }
I20260619 17:02:38.460913 10082 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95d50018b87d4e81b431ae066465c74a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 17:02:38.460937 10082 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95d50018b87d4e81b431ae066465c74a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 17:02:38.461006 10082 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95d50018b87d4e81b431ae066465c74a [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:38.461627 10082 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95d50018b87d4e81b431ae066465c74a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95d50018b87d4e81b431ae066465c74a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37353 } }
I20260619 17:02:38.461719 10082 leader_election.cc:302] T 00000000000000000000000000000000 P 95d50018b87d4e81b431ae066465c74a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95d50018b87d4e81b431ae066465c74a; no voters:
I20260619 17:02:38.461972 10082 leader_election.cc:288] T 00000000000000000000000000000000 P 95d50018b87d4e81b431ae066465c74a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 17:02:38.462050 10085 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95d50018b87d4e81b431ae066465c74a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:02:38.462172 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 10014
I20260619 17:02:38.462221 10085 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95d50018b87d4e81b431ae066465c74a [term 1 LEADER]: Becoming Leader. State: Replica: 95d50018b87d4e81b431ae066465c74a, State: Running, Role: LEADER
I20260619 17:02:38.462258 10082 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95d50018b87d4e81b431ae066465c74a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:02:38.462287 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 17:02:38.462318 10085 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95d50018b87d4e81b431ae066465c74a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95d50018b87d4e81b431ae066465c74a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37353 } }
I20260619 17:02:38.462481 21406 external_mini_cluster.cc:1428] Setting key 52448550052d773b071be9ce0620d71f
I20260619 17:02:38.462721 10085 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95d50018b87d4e81b431ae066465c74a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95d50018b87d4e81b431ae066465c74a. Latest consensus state: current_term: 1 leader_uuid: "95d50018b87d4e81b431ae066465c74a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95d50018b87d4e81b431ae066465c74a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37353 } } }
I20260619 17:02:38.462806 10085 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95d50018b87d4e81b431ae066465c74a [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:38.463074 10089 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 17:02:38.463680 10089 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 17:02:38.463688 10086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95d50018b87d4e81b431ae066465c74a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "95d50018b87d4e81b431ae066465c74a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95d50018b87d4e81b431ae066465c74a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37353 } } }
I20260619 17:02:38.463948 10086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95d50018b87d4e81b431ae066465c74a [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:38.465548 10089 catalog_manager.cc:1347] Generated new cluster ID: 2de242b3fd054fbb83ad2672f4bce59a
I20260619 17:02:38.465603 10089 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 17:02:38.467590 10091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:38.463061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.139:42555 (local address 127.0.0.1:37353)
0619 17:02:38.464392 (+ 1331us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:38.464401 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:38.464680 (+ 279us) server_negotiation.cc:407] Connection header received
0619 17:02:38.464809 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:38.464817 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:38.464851 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:38.465007 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:38.467124 (+ 2117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:38.467127 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:38.467128 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:38.467132 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:38.467168 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:38.467180 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:38.467280 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:38.467346 (+ 66us) server_negotiation.cc:299] Negotiation successful
0619 17:02:38.467404 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1279,"thread_start_us":69,"threads_started":1}
I20260619 17:02:38.469326 21406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 17:02:38.472150 10091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:38.470888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.139:53087 (local address 127.0.0.1:37353)
0619 17:02:38.470929 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:38.470931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:38.471098 (+ 167us) server_negotiation.cc:407] Connection header received
0619 17:02:38.471163 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:38.471164 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:38.471172 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:38.471222 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:38.471891 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:38.471893 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:38.471893 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:38.471898 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:38.471915 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:38.471920 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:38.471976 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:38.472045 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 17:02:38.472063 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 17:02:38.486225 10089 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 17:02:38.486467 10089 catalog_manager.cc:1504] Loading token signing keys...
I20260619 17:02:38.492395 10089 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 95d50018b87d4e81b431ae066465c74a: Generated new TSK 0
I20260619 17:02:38.492630 10089 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 17:02:38.496855 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 10014
2026-06-19T17:02:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1367 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 17:02:38 dist-test-slave-vnbj krb5kdc[10113](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 17:02:38 dist-test-slave-vnbj krb5kdc[10113](info): set up 2 sockets
Jun 19 17:02:38 dist-test-slave-vnbj krb5kdc[10113](info): commencing operation
krb5kdc: starting...
W20260619 17:02:39.607328 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.092s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 17:02:39 dist-test-slave-vnbj krb5kdc[10113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T17:02:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T17:02:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 17:02:39.743525 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37517
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37517
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:02:39.836318 10126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:39.836622 10126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:39.836654 10126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:39.838279 10126 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:02:39.838371 10126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:39.838388 10126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:39.838411 10126 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:02:39.838425 10126 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:02:39.840832 10126 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37517
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10126
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:39.841348 10126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:39.841898 10126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:39.848038 10132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:39.848083 10134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:39.848088 10131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:39.848313 10126 server_base.cc:1034] running on GCE node
I20260619 17:02:39.848654 10126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:39.849093 10126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:39.850337 10126 hybrid_clock.cc:648] HybridClock initialized: now 1781888559850286 us; error 74 us; skew 500 ppm
Jun 19 17:02:39 dist-test-slave-vnbj krb5kdc[10113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888559, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:39.853591 10126 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 17:02:39.854240 10126 webserver.cc:460] Webserver started at http://127.0.0.1:46171/ using document root <none> and password file <none>
I20260619 17:02:39.854537 10126 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:39.854597 10126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:39.854756 10126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:39.856161 10126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "349cd2bf3eaf40f197c0403f9c8e646b"
format_stamp: "Formatted at 2026-06-19 17:02:39 on dist-test-slave-vnbj"
server_key: "217522793b83a924d69d05e8cd0e8087"
server_key_iv: "0257ed0c0c173b56621030622d25c042"
server_key_version: "encryptionkey@0"
I20260619 17:02:39.856587 10126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "349cd2bf3eaf40f197c0403f9c8e646b"
format_stamp: "Formatted at 2026-06-19 17:02:39 on dist-test-slave-vnbj"
server_key: "217522793b83a924d69d05e8cd0e8087"
server_key_iv: "0257ed0c0c173b56621030622d25c042"
server_key_version: "encryptionkey@0"
I20260619 17:02:39.858268 10126 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260619 17:02:39.859336 10141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:39.859663 10126 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 17:02:39.859760 10126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "349cd2bf3eaf40f197c0403f9c8e646b"
format_stamp: "Formatted at 2026-06-19 17:02:39 on dist-test-slave-vnbj"
server_key: "217522793b83a924d69d05e8cd0e8087"
server_key_iv: "0257ed0c0c173b56621030622d25c042"
server_key_version: "encryptionkey@0"
I20260619 17:02:39.859957 10126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:39.872193 10126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:39.874985 10126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:39.875123 10126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:39.879298 10126 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37517
I20260619 17:02:39.879334 10193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37517 every 8 connection(s)
I20260619 17:02:39.879774 10126 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 17:02:39.880761 10194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:39.883307 10194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 349cd2bf3eaf40f197c0403f9c8e646b: Bootstrap starting.
I20260619 17:02:39.883889 10194 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 349cd2bf3eaf40f197c0403f9c8e646b: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:39.884141 10194 log.cc:824] T 00000000000000000000000000000000 P 349cd2bf3eaf40f197c0403f9c8e646b: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:39.885115 10194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 349cd2bf3eaf40f197c0403f9c8e646b: No bootstrap required, opened a new log
I20260619 17:02:39.886924 10194 raft_consensus.cc:348] T 00000000000000000000000000000000 P 349cd2bf3eaf40f197c0403f9c8e646b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349cd2bf3eaf40f197c0403f9c8e646b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37517 } }
I20260619 17:02:39.887099 10194 raft_consensus.cc:374] T 00000000000000000000000000000000 P 349cd2bf3eaf40f197c0403f9c8e646b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:39.887125 10194 raft_consensus.cc:729] T 00000000000000000000000000000000 P 349cd2bf3eaf40f197c0403f9c8e646b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 349cd2bf3eaf40f197c0403f9c8e646b, State: Initialized, Role: FOLLOWER
I20260619 17:02:39.887269 10194 consensus_queue.cc:260] T 00000000000000000000000000000000 P 349cd2bf3eaf40f197c0403f9c8e646b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349cd2bf3eaf40f197c0403f9c8e646b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37517 } }
I20260619 17:02:39.887352 10194 raft_consensus.cc:388] T 00000000000000000000000000000000 P 349cd2bf3eaf40f197c0403f9c8e646b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 17:02:39.887377 10194 raft_consensus.cc:482] T 00000000000000000000000000000000 P 349cd2bf3eaf40f197c0403f9c8e646b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 17:02:39.887406 10194 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 349cd2bf3eaf40f197c0403f9c8e646b [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:39.887969 10194 raft_consensus.cc:504] T 00000000000000000000000000000000 P 349cd2bf3eaf40f197c0403f9c8e646b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349cd2bf3eaf40f197c0403f9c8e646b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37517 } }
I20260619 17:02:39.888083 10194 leader_election.cc:302] T 00000000000000000000000000000000 P 349cd2bf3eaf40f197c0403f9c8e646b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 349cd2bf3eaf40f197c0403f9c8e646b; no voters:
I20260619 17:02:39.888248 10194 leader_election.cc:288] T 00000000000000000000000000000000 P 349cd2bf3eaf40f197c0403f9c8e646b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 17:02:39.888327 10197 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 349cd2bf3eaf40f197c0403f9c8e646b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:02:39.888530 10197 raft_consensus.cc:686] T 00000000000000000000000000000000 P 349cd2bf3eaf40f197c0403f9c8e646b [term 1 LEADER]: Becoming Leader. State: Replica: 349cd2bf3eaf40f197c0403f9c8e646b, State: Running, Role: LEADER
I20260619 17:02:39.888558 10194 sys_catalog.cc:564] T 00000000000000000000000000000000 P 349cd2bf3eaf40f197c0403f9c8e646b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:02:39.888700 10197 consensus_queue.cc:237] T 00000000000000000000000000000000 P 349cd2bf3eaf40f197c0403f9c8e646b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349cd2bf3eaf40f197c0403f9c8e646b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37517 } }
I20260619 17:02:39.889183 10199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 349cd2bf3eaf40f197c0403f9c8e646b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 349cd2bf3eaf40f197c0403f9c8e646b. Latest consensus state: current_term: 1 leader_uuid: "349cd2bf3eaf40f197c0403f9c8e646b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349cd2bf3eaf40f197c0403f9c8e646b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37517 } } }
I20260619 17:02:39.889333 10199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 349cd2bf3eaf40f197c0403f9c8e646b [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:39.889209 10198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 349cd2bf3eaf40f197c0403f9c8e646b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "349cd2bf3eaf40f197c0403f9c8e646b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349cd2bf3eaf40f197c0403f9c8e646b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37517 } } }
I20260619 17:02:39.889559 10201 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 17:02:39.889596 10198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 349cd2bf3eaf40f197c0403f9c8e646b [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:39.889866 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 10126
I20260619 17:02:39.889997 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 17:02:39.890188 21406 external_mini_cluster.cc:1428] Setting key 0b5f085311a9830efcb72fc2e724aaad
I20260619 17:02:39.891068 10201 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 17:02:39.892694 10201 catalog_manager.cc:1347] Generated new cluster ID: 93c2f281834d44ebb33dcb281cc7f51f
I20260619 17:02:39.892747 10201 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 17:02:39 dist-test-slave-vnbj krb5kdc[10113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 17:02:39.903520 10216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:39.890744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.139:51003 (local address 127.0.0.1:37517)
0619 17:02:39.890936 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:39.890948 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:39.891831 (+ 883us) server_negotiation.cc:407] Connection header received
0619 17:02:39.891988 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:39.891994 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:39.892080 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:39.892228 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:39.893164 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:39.894592 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:39.895755 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:39.896557 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:39.896680 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:39.899693 (+ 3013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:39.899698 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:39.899699 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:39.899713 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:39.901747 (+ 2034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:39.902203 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:39.902206 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:39.902206 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:39.902325 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:39.902521 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:39.902522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:39.902522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:39.902827 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:39.903010 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:39.903155 (+ 145us) server_negotiation.cc:299] Negotiation successful
0619 17:02:39.903269 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":65,"threads_started":1}
I20260619 17:02:39.904490 21406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 17:02:39.914690 10216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:39.906075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.139:36287 (local address 127.0.0.1:37517)
0619 17:02:39.906114 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:39.906116 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:39.906227 (+ 111us) server_negotiation.cc:407] Connection header received
0619 17:02:39.906274 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:39.906275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:39.906286 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:39.906366 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:39.907071 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:39.908021 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:39.909099 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:39.909477 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:39.909547 (+ 70us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:39.911203 (+ 1656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:39.911204 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:39.911205 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:39.911213 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:39.913176 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:39.913675 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:39.913676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:39.913677 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:39.913751 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:39.914012 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:39.914013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:39.914013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:39.914144 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:39.914283 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:39.914458 (+ 175us) server_negotiation.cc:299] Negotiation successful
0619 17:02:39.914565 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 17:02:39.914872 10201 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 17:02:39.915656 10201 catalog_manager.cc:1504] Loading token signing keys...
I20260619 17:02:39.920316 10201 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 349cd2bf3eaf40f197c0403f9c8e646b: Generated new TSK 0
I20260619 17:02:39.920549 10201 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 17:02:39.922823 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 10126
W20260619 17:02:39.926746 10111 connection.cc:535] client connection to 127.0.0.1:37517 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37517): Connection reset by peer (error 104)
2026-06-19T17:02:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1425 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 17:02:39 dist-test-slave-vnbj krb5kdc[10225](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 17:02:39 dist-test-slave-vnbj krb5kdc[10225](info): set up 2 sockets
Jun 19 17:02:39 dist-test-slave-vnbj krb5kdc[10225](info): commencing operation
krb5kdc: starting...
W20260619 17:02:41.031924 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 17:02:41 dist-test-slave-vnbj krb5kdc[10225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T17:02:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T17:02:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 17:02:41.179653 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40557
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40557
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:02:41.265794 10238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:41.266108 10238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:41.266155 10238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:41.267613 10238 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:02:41.267704 10238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:41.267727 10238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:41.267748 10238 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:02:41.267774 10238 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:02:41.269587 10238 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40557
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:41.270041 10238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:41.270309 10238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:41.275910 10246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:41.275974 10243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:41.275996 10244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:41.276208 10238 server_base.cc:1034] running on GCE node
I20260619 17:02:41.276562 10238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:41.276904 10238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:41.278074 10238 hybrid_clock.cc:648] HybridClock initialized: now 1781888561278046 us; error 41 us; skew 500 ppm
Jun 19 17:02:41 dist-test-slave-vnbj krb5kdc[10225](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888561, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:41.281080 10238 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 17:02:41.281689 10238 webserver.cc:460] Webserver started at http://127.0.0.1:39581/ using document root <none> and password file <none>
I20260619 17:02:41.281915 10238 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:41.281963 10238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:41.282068 10238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:41.283489 10238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "57f2add5eb474f178d9104df52c42edf"
format_stamp: "Formatted at 2026-06-19 17:02:41 on dist-test-slave-vnbj"
server_key: "106fafd5b8a5256c4babc9b97c664df2"
server_key_iv: "664bf8d9b71d2a24f98bd410e2773563"
server_key_version: "encryptionkey@0"
I20260619 17:02:41.283851 10238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "57f2add5eb474f178d9104df52c42edf"
format_stamp: "Formatted at 2026-06-19 17:02:41 on dist-test-slave-vnbj"
server_key: "106fafd5b8a5256c4babc9b97c664df2"
server_key_iv: "664bf8d9b71d2a24f98bd410e2773563"
server_key_version: "encryptionkey@0"
I20260619 17:02:41.285585 10238 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260619 17:02:41.286587 10253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:41.287003 10238 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:41.287133 10238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "57f2add5eb474f178d9104df52c42edf"
format_stamp: "Formatted at 2026-06-19 17:02:41 on dist-test-slave-vnbj"
server_key: "106fafd5b8a5256c4babc9b97c664df2"
server_key_iv: "664bf8d9b71d2a24f98bd410e2773563"
server_key_version: "encryptionkey@0"
I20260619 17:02:41.287225 10238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:41.299167 10238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:41.301750 10238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:41.302119 10238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:41.307157 10238 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40557
I20260619 17:02:41.307257 10305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40557 every 8 connection(s)
I20260619 17:02:41.307682 10238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 17:02:41.308808 10306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:41.311630 10306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57f2add5eb474f178d9104df52c42edf: Bootstrap starting.
I20260619 17:02:41.312351 10306 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57f2add5eb474f178d9104df52c42edf: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:41.312695 10306 log.cc:824] T 00000000000000000000000000000000 P 57f2add5eb474f178d9104df52c42edf: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:41.313668 10306 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57f2add5eb474f178d9104df52c42edf: No bootstrap required, opened a new log
I20260619 17:02:41.315438 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 10238
I20260619 17:02:41.315544 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 17:02:41.315739 21406 external_mini_cluster.cc:1428] Setting key 3a4585ff928f0f466181e393564c67d8
I20260619 17:02:41.315879 10306 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57f2add5eb474f178d9104df52c42edf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57f2add5eb474f178d9104df52c42edf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40557 } }
I20260619 17:02:41.316082 10306 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57f2add5eb474f178d9104df52c42edf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:41.316174 10306 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57f2add5eb474f178d9104df52c42edf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57f2add5eb474f178d9104df52c42edf, State: Initialized, Role: FOLLOWER
I20260619 17:02:41.316391 10306 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57f2add5eb474f178d9104df52c42edf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57f2add5eb474f178d9104df52c42edf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40557 } }
I20260619 17:02:41.316509 10306 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57f2add5eb474f178d9104df52c42edf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 17:02:41.316550 10306 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57f2add5eb474f178d9104df52c42edf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 17:02:41.316653 10306 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57f2add5eb474f178d9104df52c42edf [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:41.317749 10306 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57f2add5eb474f178d9104df52c42edf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57f2add5eb474f178d9104df52c42edf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40557 } }
I20260619 17:02:41.317898 10306 leader_election.cc:302] T 00000000000000000000000000000000 P 57f2add5eb474f178d9104df52c42edf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57f2add5eb474f178d9104df52c42edf; no voters:
I20260619 17:02:41.318121 10306 leader_election.cc:288] T 00000000000000000000000000000000 P 57f2add5eb474f178d9104df52c42edf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 17:02:41.318152 10311 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57f2add5eb474f178d9104df52c42edf [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:02:41.318272 10311 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57f2add5eb474f178d9104df52c42edf [term 1 LEADER]: Becoming Leader. State: Replica: 57f2add5eb474f178d9104df52c42edf, State: Running, Role: LEADER
I20260619 17:02:41.318439 10311 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57f2add5eb474f178d9104df52c42edf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57f2add5eb474f178d9104df52c42edf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40557 } }
I20260619 17:02:41.318954 10312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57f2add5eb474f178d9104df52c42edf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57f2add5eb474f178d9104df52c42edf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57f2add5eb474f178d9104df52c42edf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40557 } } }
I20260619 17:02:41.319013 10313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57f2add5eb474f178d9104df52c42edf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57f2add5eb474f178d9104df52c42edf. Latest consensus state: current_term: 1 leader_uuid: "57f2add5eb474f178d9104df52c42edf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57f2add5eb474f178d9104df52c42edf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40557 } } }
I20260619 17:02:41.319105 10312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57f2add5eb474f178d9104df52c42edf [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:41.319121 10313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57f2add5eb474f178d9104df52c42edf [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:41.319373 10315 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 17:02:41.318449 10306 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57f2add5eb474f178d9104df52c42edf [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:02:41.319986 10315 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 17:02:41.321839 10315 catalog_manager.cc:1347] Generated new cluster ID: 3d170d4271bf4884aaaa76f565de529a
I20260619 17:02:41.321899 10315 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 17:02:41.322297 10310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:41.316487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.139:34015 (local address 127.0.0.1:40557)
0619 17:02:41.316723 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:41.316734 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:41.316784 (+ 50us) server_negotiation.cc:407] Connection header received
0619 17:02:41.316926 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:41.316932 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:41.316983 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:41.317132 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:41.317342 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:41.318425 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:41.319747 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:41.320350 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:41.320471 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:41.321444 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:41.321448 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:41.321448 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:41.321691 (+ 243us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:41.321731 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:41.321748 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:41.321835 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:41.321997 (+ 162us) server_negotiation.cc:299] Negotiation successful
0619 17:02:41.322124 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":115,"threads_started":1}
I20260619 17:02:41.323310 21406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 17:02:41.327947 10310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:41.324720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.139:43485 (local address 127.0.0.1:40557)
0619 17:02:41.324762 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:41.324764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:41.324905 (+ 141us) server_negotiation.cc:407] Connection header received
0619 17:02:41.324973 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:41.324974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:41.324985 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:41.325032 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:41.325186 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:41.325683 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:41.326453 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:41.326693 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:41.326778 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:41.327648 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:41.327649 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:41.327650 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:41.327656 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:41.327673 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:41.327679 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:41.327739 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:41.327813 (+ 74us) server_negotiation.cc:299] Negotiation successful
0619 17:02:41.327828 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 17:02:41.343787 10315 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 17:02:41.344664 10315 catalog_manager.cc:1504] Loading token signing keys...
I20260619 17:02:41.356060 10315 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57f2add5eb474f178d9104df52c42edf: Generated new TSK 0
I20260619 17:02:41.356343 10315 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 17:02:41.358877 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 10238
W20260619 17:02:41.363986 10223 connection.cc:535] client connection to 127.0.0.1:40557 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40557): Connection reset by peer (error 104)
2026-06-19T17:02:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1437 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 17:02:41 dist-test-slave-vnbj krb5kdc[10337](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 17:02:41 dist-test-slave-vnbj krb5kdc[10337](info): set up 2 sockets
Jun 19 17:02:41 dist-test-slave-vnbj krb5kdc[10337](info): commencing operation
krb5kdc: starting...
W20260619 17:02:42.423192 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 17:02:42 dist-test-slave-vnbj krb5kdc[10337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888562, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T17:02:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T17:02:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 17:02:42.564375 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46711
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46711
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:02:42.648100 10350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:42.648355 10350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:42.648382 10350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:42.649624 10350 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:02:42.649675 10350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:42.649690 10350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:42.649704 10350 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:02:42.649717 10350 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:02:42.651463 10350 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46711
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888301345144-21406-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46711
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:42.651703 10350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:42.651893 10350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:42.657155 10355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:42.657191 10356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:42.657155 10358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:42.657552 10350 server_base.cc:1034] running on GCE node
I20260619 17:02:42.657819 10350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:42.658187 10350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:42.659366 10350 hybrid_clock.cc:648] HybridClock initialized: now 1781888562659337 us; error 44 us; skew 500 ppm
Jun 19 17:02:42 dist-test-slave-vnbj krb5kdc[10337](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888562, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:42.662370 10350 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 17:02:42.662973 10350 webserver.cc:460] Webserver started at http://127.0.0.1:35921/ using document root <none> and password file <none>
I20260619 17:02:42.663213 10350 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:42.663259 10350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:42.663359 10350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:42.664433 10350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "c3b455bf810a45508b9522eafa9afe5b"
format_stamp: "Formatted at 2026-06-19 17:02:42 on dist-test-slave-vnbj"
server_key: "d9efdb1ea2f92921f294d4d7adb918ad"
server_key_iv: "ce399881367ef268411dacb1d6949fca"
server_key_version: "encryptionkey@0"
I20260619 17:02:42.664744 10350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "c3b455bf810a45508b9522eafa9afe5b"
format_stamp: "Formatted at 2026-06-19 17:02:42 on dist-test-slave-vnbj"
server_key: "d9efdb1ea2f92921f294d4d7adb918ad"
server_key_iv: "ce399881367ef268411dacb1d6949fca"
server_key_version: "encryptionkey@0"
I20260619 17:02:42.666147 10350 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 17:02:42.667141 10365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:42.667454 10350 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:42.667565 10350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "c3b455bf810a45508b9522eafa9afe5b"
format_stamp: "Formatted at 2026-06-19 17:02:42 on dist-test-slave-vnbj"
server_key: "d9efdb1ea2f92921f294d4d7adb918ad"
server_key_iv: "ce399881367ef268411dacb1d6949fca"
server_key_version: "encryptionkey@0"
I20260619 17:02:42.667647 10350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:42.671461 10350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:42.674248 10350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:42.674429 10350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:42.679117 10350 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46711
I20260619 17:02:42.679198 10417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46711 every 8 connection(s)
I20260619 17:02:42.679557 10350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 17:02:42.680179 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 10350
I20260619 17:02:42.680332 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 17:02:42.680593 21406 external_mini_cluster.cc:1428] Setting key f3c5f13488d3030bd8befefd87933287
I20260619 17:02:42.680904 10418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:42.684028 10421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:42.681317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.139:59199 (local address 127.0.0.1:46711)
0619 17:02:42.681518 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:42.681526 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:42.681560 (+ 34us) server_negotiation.cc:407] Connection header received
0619 17:02:42.682385 (+ 825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:42.682394 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:42.682434 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:42.682567 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:42.683455 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:42.683459 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:42.683459 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:42.683463 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:42.683507 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:42.683521 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:42.683587 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:42.683669 (+ 82us) server_negotiation.cc:299] Negotiation successful
0619 17:02:42.683761 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":92,"threads_started":1}
I20260619 17:02:42.684648 10418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3b455bf810a45508b9522eafa9afe5b: Bootstrap starting.
I20260619 17:02:42.685346 21406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 17:02:42.685474 10418 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3b455bf810a45508b9522eafa9afe5b: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:42.685770 10418 log.cc:824] T 00000000000000000000000000000000 P c3b455bf810a45508b9522eafa9afe5b: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:42.686594 10418 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3b455bf810a45508b9522eafa9afe5b: No bootstrap required, opened a new log
I20260619 17:02:42.688673 10421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:42.687175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.139:55087 (local address 127.0.0.1:46711)
0619 17:02:42.687248 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:42.687298 (+ 50us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:42.687457 (+ 159us) server_negotiation.cc:407] Connection header received
0619 17:02:42.687586 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:42.687587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:42.687593 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:42.687656 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:42.688393 (+ 737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:42.688395 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:42.688396 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:42.688401 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:42.688416 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:42.688421 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:42.688512 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:42.688587 (+ 75us) server_negotiation.cc:299] Negotiation successful
0619 17:02:42.688602 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 17:02:42.689023 10418 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3b455bf810a45508b9522eafa9afe5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b455bf810a45508b9522eafa9afe5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46711 } }
I20260619 17:02:42.689234 10418 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3b455bf810a45508b9522eafa9afe5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:42.689298 10418 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3b455bf810a45508b9522eafa9afe5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3b455bf810a45508b9522eafa9afe5b, State: Initialized, Role: FOLLOWER
I20260619 17:02:42.689488 10418 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3b455bf810a45508b9522eafa9afe5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b455bf810a45508b9522eafa9afe5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46711 } }
I20260619 17:02:42.689599 10418 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3b455bf810a45508b9522eafa9afe5b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 17:02:42.689647 10418 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3b455bf810a45508b9522eafa9afe5b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 17:02:42.689700 10418 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3b455bf810a45508b9522eafa9afe5b [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:42.690320 10418 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3b455bf810a45508b9522eafa9afe5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b455bf810a45508b9522eafa9afe5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46711 } }
I20260619 17:02:42.690457 10418 leader_election.cc:302] T 00000000000000000000000000000000 P c3b455bf810a45508b9522eafa9afe5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3b455bf810a45508b9522eafa9afe5b; no voters:
I20260619 17:02:42.690656 10418 leader_election.cc:288] T 00000000000000000000000000000000 P c3b455bf810a45508b9522eafa9afe5b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 17:02:42.691025 10428 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3b455bf810a45508b9522eafa9afe5b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:02:42.691218 10428 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3b455bf810a45508b9522eafa9afe5b [term 1 LEADER]: Becoming Leader. State: Replica: c3b455bf810a45508b9522eafa9afe5b, State: Running, Role: LEADER
I20260619 17:02:42.691231 10418 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3b455bf810a45508b9522eafa9afe5b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:02:42.691323 10428 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3b455bf810a45508b9522eafa9afe5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b455bf810a45508b9522eafa9afe5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46711 } }
I20260619 17:02:42.691709 10429 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3b455bf810a45508b9522eafa9afe5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3b455bf810a45508b9522eafa9afe5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b455bf810a45508b9522eafa9afe5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46711 } } }
I20260619 17:02:42.691912 10430 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3b455bf810a45508b9522eafa9afe5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3b455bf810a45508b9522eafa9afe5b. Latest consensus state: current_term: 1 leader_uuid: "c3b455bf810a45508b9522eafa9afe5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3b455bf810a45508b9522eafa9afe5b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46711 } } }
I20260619 17:02:42.692020 10429 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3b455bf810a45508b9522eafa9afe5b [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:42.692094 10430 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3b455bf810a45508b9522eafa9afe5b [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:42.692466 10437 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 17:02:42.693006 10437 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 17:02:42.694643 10437 catalog_manager.cc:1347] Generated new cluster ID: 20ecb32b631f414c82e1d700eeab96cd
I20260619 17:02:42.694703 10437 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 17:02:42.721433 10437 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 17:02:42.721669 10437 catalog_manager.cc:1504] Loading token signing keys...
I20260619 17:02:42.726558 10437 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3b455bf810a45508b9522eafa9afe5b: Generated new TSK 0
I20260619 17:02:42.726817 10437 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 17:02:42.736021 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 10350
2026-06-19T17:02:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1376 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 17:02:42 dist-test-slave-vnbj krb5kdc[10449](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 17:02:42 dist-test-slave-vnbj krb5kdc[10449](info): set up 2 sockets
Jun 19 17:02:42 dist-test-slave-vnbj krb5kdc[10449](info): commencing operation
krb5kdc: starting...
W20260619 17:02:43.803216 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.048s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 17:02:43 dist-test-slave-vnbj krb5kdc[10449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T17:02:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T17:02:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.231.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
I20260619 17:02:43.944067 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:42655
--webserver_interface=127.20.231.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:39687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:42655
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:02:44.032686 10462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:44.032958 10462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:44.032987 10462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:44.034171 10462 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:02:44.034224 10462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:44.034240 10462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:44.034255 10462 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:02:44.034271 10462 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:02:44.035949 10462 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:39687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:42655
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:42655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:44.036199 10462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:44.036391 10462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:44.041831 10467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:44.041855 10470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:44.041857 10468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:44.041965 10462 server_base.cc:1034] running on GCE node
I20260619 17:02:44.042496 10462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:44.042834 10462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:44.044160 10462 hybrid_clock.cc:648] HybridClock initialized: now 1781888564044117 us; error 56 us; skew 500 ppm
Jun 19 17:02:44 dist-test-slave-vnbj krb5kdc[10449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888564, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:44.058458 10462 init.cc:377] Logged in from keytab as kudu/127.20.231.190@KRBTEST.COM (short username kudu)
I20260619 17:02:44.059161 10462 webserver.cc:460] Webserver started at http://127.20.231.190:39711/ using document root <none> and password file <none>
I20260619 17:02:44.059464 10462 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:44.059546 10462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:44.059705 10462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:44.060914 10462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "2d35726ce8d34f86bda9ad26aa097d8d"
format_stamp: "Formatted at 2026-06-19 17:02:44 on dist-test-slave-vnbj"
server_key: "d1ca92974737be9aaf34b78401f2672c"
server_key_iv: "739dde893fc06c89dd4b522c039b0523"
server_key_version: "encryptionkey@0"
I20260619 17:02:44.061316 10462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "2d35726ce8d34f86bda9ad26aa097d8d"
format_stamp: "Formatted at 2026-06-19 17:02:44 on dist-test-slave-vnbj"
server_key: "d1ca92974737be9aaf34b78401f2672c"
server_key_iv: "739dde893fc06c89dd4b522c039b0523"
server_key_version: "encryptionkey@0"
I20260619 17:02:44.063119 10462 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.001s
I20260619 17:02:44.064313 10477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:44.064625 10462 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:44.064790 10462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "2d35726ce8d34f86bda9ad26aa097d8d"
format_stamp: "Formatted at 2026-06-19 17:02:44 on dist-test-slave-vnbj"
server_key: "d1ca92974737be9aaf34b78401f2672c"
server_key_iv: "739dde893fc06c89dd4b522c039b0523"
server_key_version: "encryptionkey@0"
I20260619 17:02:44.064908 10462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:44.093125 10462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:44.096138 10462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:44.096338 10462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:44.101178 10462 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:42655
I20260619 17:02:44.101261 10529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:42655 every 8 connection(s)
I20260619 17:02:44.101676 10462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 17:02:44.102666 10530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:44.105273 10530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d35726ce8d34f86bda9ad26aa097d8d: Bootstrap starting.
I20260619 17:02:44.105888 10530 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d35726ce8d34f86bda9ad26aa097d8d: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:44.106153 10530 log.cc:824] T 00000000000000000000000000000000 P 2d35726ce8d34f86bda9ad26aa097d8d: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:44.107038 10530 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d35726ce8d34f86bda9ad26aa097d8d: No bootstrap required, opened a new log
I20260619 17:02:44.108781 10530 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d35726ce8d34f86bda9ad26aa097d8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d35726ce8d34f86bda9ad26aa097d8d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 42655 } }
I20260619 17:02:44.108955 10530 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d35726ce8d34f86bda9ad26aa097d8d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:44.108981 10530 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d35726ce8d34f86bda9ad26aa097d8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d35726ce8d34f86bda9ad26aa097d8d, State: Initialized, Role: FOLLOWER
I20260619 17:02:44.109143 10530 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d35726ce8d34f86bda9ad26aa097d8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d35726ce8d34f86bda9ad26aa097d8d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 42655 } }
I20260619 17:02:44.109234 10530 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d35726ce8d34f86bda9ad26aa097d8d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 17:02:44.109259 10530 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d35726ce8d34f86bda9ad26aa097d8d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 17:02:44.109292 10530 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d35726ce8d34f86bda9ad26aa097d8d [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:44.110235 10530 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d35726ce8d34f86bda9ad26aa097d8d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d35726ce8d34f86bda9ad26aa097d8d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 42655 } }
I20260619 17:02:44.110334 10530 leader_election.cc:302] T 00000000000000000000000000000000 P 2d35726ce8d34f86bda9ad26aa097d8d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d35726ce8d34f86bda9ad26aa097d8d; no voters:
I20260619 17:02:44.110491 10530 leader_election.cc:288] T 00000000000000000000000000000000 P 2d35726ce8d34f86bda9ad26aa097d8d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 17:02:44.110584 10533 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d35726ce8d34f86bda9ad26aa097d8d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:02:44.110760 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 10462
I20260619 17:02:44.110816 10530 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d35726ce8d34f86bda9ad26aa097d8d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:02:44.110931 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 17:02:44.110880 10533 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d35726ce8d34f86bda9ad26aa097d8d [term 1 LEADER]: Becoming Leader. State: Replica: 2d35726ce8d34f86bda9ad26aa097d8d, State: Running, Role: LEADER
I20260619 17:02:44.111042 10533 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d35726ce8d34f86bda9ad26aa097d8d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d35726ce8d34f86bda9ad26aa097d8d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 42655 } }
I20260619 17:02:44.111146 21406 external_mini_cluster.cc:1428] Setting key fbe0b8bd6d1d94b0851e9dae2bd84d06
I20260619 17:02:44.111478 10535 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d35726ce8d34f86bda9ad26aa097d8d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d35726ce8d34f86bda9ad26aa097d8d. Latest consensus state: current_term: 1 leader_uuid: "2d35726ce8d34f86bda9ad26aa097d8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d35726ce8d34f86bda9ad26aa097d8d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 42655 } } }
I20260619 17:02:44.111565 10535 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d35726ce8d34f86bda9ad26aa097d8d [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:44.111941 10534 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d35726ce8d34f86bda9ad26aa097d8d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d35726ce8d34f86bda9ad26aa097d8d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d35726ce8d34f86bda9ad26aa097d8d" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 42655 } } }
I20260619 17:02:44.112025 10534 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d35726ce8d34f86bda9ad26aa097d8d [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:44.112290 10537 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 17:02:44.112879 10537 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 17:02:44.114874 10537 catalog_manager.cc:1347] Generated new cluster ID: 8b4edb96ff2c4a5782f6a65ed6e926c3
I20260619 17:02:44.114938 10537 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Jun 19 17:02:44 dist-test-slave-vnbj krb5kdc[10449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:02:44.123160 10539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:44.111909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39803 (local address 127.20.231.190:42655)
0619 17:02:44.112114 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:44.112125 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:44.112343 (+ 218us) server_negotiation.cc:407] Connection header received
0619 17:02:44.112475 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:44.112483 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:44.112543 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:44.112695 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:44.113775 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:44.114965 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:44.115785 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:44.116257 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:44.116413 (+ 156us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:44.119110 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:44.119123 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:44.119123 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:44.119136 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:44.121096 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:44.121584 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:44.121588 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:44.121588 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:44.121732 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:44.122015 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:44.122016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:44.122017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:44.122316 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:44.122533 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:44.122667 (+ 134us) server_negotiation.cc:299] Negotiation successful
0619 17:02:44.122972 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":75,"threads_started":1}
I20260619 17:02:44.124248 21406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 17:02:44.128674 10537 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 17:02:44.129492 10537 catalog_manager.cc:1504] Loading token signing keys...
I20260619 17:02:44.133549 10539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:44.125745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45805 (local address 127.20.231.190:42655)
0619 17:02:44.125790 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:44.125792 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:44.125931 (+ 139us) server_negotiation.cc:407] Connection header received
0619 17:02:44.126013 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:44.126014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:44.126026 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:44.126088 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:44.126676 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:44.127346 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:44.128734 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:44.129019 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:44.129122 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:44.130510 (+ 1388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:44.130512 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:44.130513 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:44.130521 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:44.132173 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:44.132604 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:44.132605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:44.132606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:44.132677 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:44.132950 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:44.132951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:44.132951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:44.133079 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:44.133221 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:44.133360 (+ 139us) server_negotiation.cc:299] Negotiation successful
0619 17:02:44.133431 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":22}
I20260619 17:02:44.141379 10537 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d35726ce8d34f86bda9ad26aa097d8d: Generated new TSK 0
I20260619 17:02:44.141641 10537 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 17:02:44.143821 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 10462
W20260619 17:02:44.148461 10447 connection.cc:535] client connection to 127.20.231.190:42655 recv error: Network error: failed to read from TLS socket (remote: 127.20.231.190:42655): Connection reset by peer (error 104)
2026-06-19T17:02:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1407 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 17:02:44 dist-test-slave-vnbj krb5kdc[10561](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 17:02:44 dist-test-slave-vnbj krb5kdc[10561](info): set up 2 sockets
Jun 19 17:02:44 dist-test-slave-vnbj krb5kdc[10561](info): commencing operation
krb5kdc: starting...
W20260619 17:02:45.235260 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 17:02:45 dist-test-slave-vnbj krb5kdc[10561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T17:02:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T17:02:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.231.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
I20260619 17:02:45.379242 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:41743
--webserver_interface=127.20.231.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:41873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:41743
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:02:45.473838 10574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:45.474148 10574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:45.474179 10574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:45.475911 10574 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:02:45.476028 10574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:45.476054 10574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:45.476106 10574 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:02:45.476138 10574 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:02:45.478211 10574 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:41873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:41743
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:41743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:45.478578 10574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:45.478961 10574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 17:02:45.485885 10574 server_base.cc:1034] running on GCE node
W20260619 17:02:45.485863 10580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:45.485929 10582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:45.485847 10579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:45.486434 10574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:45.486760 10574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:45.487993 10574 hybrid_clock.cc:648] HybridClock initialized: now 1781888565487916 us; error 58 us; skew 500 ppm
Jun 19 17:02:45 dist-test-slave-vnbj krb5kdc[10561](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888565, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:45.491293 10574 init.cc:377] Logged in from keytab as kudu/127.20.231.190@KRBTEST.COM (short username kudu)
I20260619 17:02:45.491901 10574 webserver.cc:460] Webserver started at http://127.20.231.190:34221/ using document root <none> and password file <none>
I20260619 17:02:45.492197 10574 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:45.492275 10574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:45.492444 10574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:45.493672 10574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "00ce34883a2b410fabe4ff7d27720aba"
format_stamp: "Formatted at 2026-06-19 17:02:45 on dist-test-slave-vnbj"
server_key: "67700c53b8e3106869faef1cc8121b18"
server_key_iv: "fb2a66d459119f4ea1f3e8b893164840"
server_key_version: "encryptionkey@0"
I20260619 17:02:45.494078 10574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "00ce34883a2b410fabe4ff7d27720aba"
format_stamp: "Formatted at 2026-06-19 17:02:45 on dist-test-slave-vnbj"
server_key: "67700c53b8e3106869faef1cc8121b18"
server_key_iv: "fb2a66d459119f4ea1f3e8b893164840"
server_key_version: "encryptionkey@0"
I20260619 17:02:45.495726 10574 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260619 17:02:45.496711 10589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:45.496992 10574 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:45.497128 10574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "00ce34883a2b410fabe4ff7d27720aba"
format_stamp: "Formatted at 2026-06-19 17:02:45 on dist-test-slave-vnbj"
server_key: "67700c53b8e3106869faef1cc8121b18"
server_key_iv: "fb2a66d459119f4ea1f3e8b893164840"
server_key_version: "encryptionkey@0"
I20260619 17:02:45.497225 10574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:45.508090 10574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:45.511018 10574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:45.511212 10574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:45.515673 10574 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:41743
I20260619 17:02:45.516019 10641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:41743 every 8 connection(s)
I20260619 17:02:45.516191 10574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 17:02:45.517011 10642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:45.519824 10642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00ce34883a2b410fabe4ff7d27720aba: Bootstrap starting.
I20260619 17:02:45.520488 10642 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00ce34883a2b410fabe4ff7d27720aba: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:45.520823 10642 log.cc:824] T 00000000000000000000000000000000 P 00ce34883a2b410fabe4ff7d27720aba: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:45.521708 10642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00ce34883a2b410fabe4ff7d27720aba: No bootstrap required, opened a new log
I20260619 17:02:45.523722 10642 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00ce34883a2b410fabe4ff7d27720aba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ce34883a2b410fabe4ff7d27720aba" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41743 } }
I20260619 17:02:45.523947 10642 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00ce34883a2b410fabe4ff7d27720aba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:45.523974 10642 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00ce34883a2b410fabe4ff7d27720aba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00ce34883a2b410fabe4ff7d27720aba, State: Initialized, Role: FOLLOWER
I20260619 17:02:45.524138 10642 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00ce34883a2b410fabe4ff7d27720aba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ce34883a2b410fabe4ff7d27720aba" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41743 } }
I20260619 17:02:45.524214 10642 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00ce34883a2b410fabe4ff7d27720aba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 17:02:45.524258 10642 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00ce34883a2b410fabe4ff7d27720aba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 17:02:45.524318 10642 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00ce34883a2b410fabe4ff7d27720aba [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:45.525698 10642 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00ce34883a2b410fabe4ff7d27720aba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ce34883a2b410fabe4ff7d27720aba" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41743 } }
I20260619 17:02:45.525884 10642 leader_election.cc:302] T 00000000000000000000000000000000 P 00ce34883a2b410fabe4ff7d27720aba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00ce34883a2b410fabe4ff7d27720aba; no voters:
I20260619 17:02:45.526090 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 10574
I20260619 17:02:45.526197 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 17:02:45.526262 10642 leader_election.cc:288] T 00000000000000000000000000000000 P 00ce34883a2b410fabe4ff7d27720aba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 17:02:45.526456 21406 external_mini_cluster.cc:1428] Setting key 4d5a267992c93a4243d0c536e2383132
I20260619 17:02:45.526435 10645 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00ce34883a2b410fabe4ff7d27720aba [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:02:45.526603 10645 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00ce34883a2b410fabe4ff7d27720aba [term 1 LEADER]: Becoming Leader. State: Replica: 00ce34883a2b410fabe4ff7d27720aba, State: Running, Role: LEADER
I20260619 17:02:45.526744 10645 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00ce34883a2b410fabe4ff7d27720aba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ce34883a2b410fabe4ff7d27720aba" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41743 } }
I20260619 17:02:45.526595 10642 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00ce34883a2b410fabe4ff7d27720aba [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:02:45.527212 10646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ce34883a2b410fabe4ff7d27720aba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00ce34883a2b410fabe4ff7d27720aba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ce34883a2b410fabe4ff7d27720aba" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41743 } } }
I20260619 17:02:45.527311 10646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ce34883a2b410fabe4ff7d27720aba [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:45.527496 10647 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00ce34883a2b410fabe4ff7d27720aba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00ce34883a2b410fabe4ff7d27720aba. Latest consensus state: current_term: 1 leader_uuid: "00ce34883a2b410fabe4ff7d27720aba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00ce34883a2b410fabe4ff7d27720aba" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 41743 } } }
I20260619 17:02:45.527557 10651 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 17:02:45.527582 10647 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00ce34883a2b410fabe4ff7d27720aba [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:45.528126 10651 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 17:02:45.530040 10651 catalog_manager.cc:1347] Generated new cluster ID: ce9d9e0809e64922ba4c0f9c79ab4479
I20260619 17:02:45.530112 10651 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 17:02:45.533398 10650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:45.527061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45905 (local address 127.20.231.190:41743)
0619 17:02:45.527438 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:45.527449 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:45.527488 (+ 39us) server_negotiation.cc:407] Connection header received
0619 17:02:45.527749 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:45.527760 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:45.527839 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:45.528401 (+ 562us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:45.528550 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:45.530025 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:45.531106 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:45.531597 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:45.531762 (+ 165us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:45.532619 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:45.532623 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:45.532624 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:45.532835 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:45.532872 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:45.532885 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:45.533014 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:45.533106 (+ 92us) server_negotiation.cc:299] Negotiation successful
0619 17:02:45.533145 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":77,"threads_started":1}
I20260619 17:02:45.535050 21406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 17:02:45.542738 10650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:45.539325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39163 (local address 127.20.231.190:41743)
0619 17:02:45.539373 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:45.539376 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:45.539558 (+ 182us) server_negotiation.cc:407] Connection header received
0619 17:02:45.539649 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:45.539651 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:45.539663 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:45.539711 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:45.539894 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:45.540484 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:45.541293 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:45.541532 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:45.541613 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:45.542431 (+ 818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:45.542432 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:45.542432 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:45.542438 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:45.542454 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:45.542459 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:45.542533 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:45.542607 (+ 74us) server_negotiation.cc:299] Negotiation successful
0619 17:02:45.542622 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 17:02:45.555130 10651 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 17:02:45.556166 10651 catalog_manager.cc:1504] Loading token signing keys...
I20260619 17:02:45.563715 10651 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00ce34883a2b410fabe4ff7d27720aba: Generated new TSK 0
I20260619 17:02:45.564034 10651 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 17:02:45.575937 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 10574
W20260619 17:02:45.581686 10559 connection.cc:535] client connection to 127.20.231.190:41743 recv error: Network error: failed to read from TLS socket (remote: 127.20.231.190:41743): Connection reset by peer (error 104)
2026-06-19T17:02:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1432 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 17:02:45 dist-test-slave-vnbj krb5kdc[10673](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 17:02:45 dist-test-slave-vnbj krb5kdc[10673](info): set up 2 sockets
Jun 19 17:02:45 dist-test-slave-vnbj krb5kdc[10673](info): commencing operation
krb5kdc: starting...
W20260619 17:02:46.679354 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 17:02:46 dist-test-slave-vnbj krb5kdc[10673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888566, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T17:02:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T17:02:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.231.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
I20260619 17:02:46.827741 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:32901
--webserver_interface=127.20.231.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:34281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:32901
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:02:46.916558 10686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:46.916808 10686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:46.916836 10686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:46.918131 10686 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:02:46.918195 10686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:46.918211 10686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:46.918224 10686 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:02:46.918237 10686 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:02:46.920117 10686 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:34281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:32901
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:32901
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10686
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:46.920441 10686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:46.920712 10686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:46.926452 10694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:46.926442 10691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:46.926473 10692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:46.926705 10686 server_base.cc:1034] running on GCE node
I20260619 17:02:46.927343 10686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:46.927718 10686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:46.928994 10686 hybrid_clock.cc:648] HybridClock initialized: now 1781888566928966 us; error 46 us; skew 500 ppm
Jun 19 17:02:46 dist-test-slave-vnbj krb5kdc[10673](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888566, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:46.937724 10686 init.cc:377] Logged in from keytab as kudu/127.20.231.190@KRBTEST.COM (short username kudu)
I20260619 17:02:46.938387 10686 webserver.cc:460] Webserver started at http://127.20.231.190:35871/ using document root <none> and password file <none>
I20260619 17:02:46.938650 10686 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:46.938726 10686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:46.938954 10686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:46.940145 10686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "491739bd674d493195daacf9afe48cb1"
format_stamp: "Formatted at 2026-06-19 17:02:46 on dist-test-slave-vnbj"
server_key: "47f006b2fd400f473c3b29fc638dfab3"
server_key_iv: "6b241ecc4a95edf2e29e76c86bd72018"
server_key_version: "encryptionkey@0"
I20260619 17:02:46.940550 10686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "491739bd674d493195daacf9afe48cb1"
format_stamp: "Formatted at 2026-06-19 17:02:46 on dist-test-slave-vnbj"
server_key: "47f006b2fd400f473c3b29fc638dfab3"
server_key_iv: "6b241ecc4a95edf2e29e76c86bd72018"
server_key_version: "encryptionkey@0"
I20260619 17:02:46.942098 10686 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 17:02:46.943269 10701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:46.943588 10686 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:46.943701 10686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "491739bd674d493195daacf9afe48cb1"
format_stamp: "Formatted at 2026-06-19 17:02:46 on dist-test-slave-vnbj"
server_key: "47f006b2fd400f473c3b29fc638dfab3"
server_key_iv: "6b241ecc4a95edf2e29e76c86bd72018"
server_key_version: "encryptionkey@0"
I20260619 17:02:46.943805 10686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:46.947587 10686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:46.950170 10686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:46.950299 10686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:46.955252 10686 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:32901
I20260619 17:02:46.955387 10753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:32901 every 8 connection(s)
I20260619 17:02:46.955761 10686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 17:02:46.956897 10754 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:46.959882 10754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 491739bd674d493195daacf9afe48cb1: Bootstrap starting.
I20260619 17:02:46.960544 10754 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 491739bd674d493195daacf9afe48cb1: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:46.960901 10754 log.cc:824] T 00000000000000000000000000000000 P 491739bd674d493195daacf9afe48cb1: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:46.961860 10754 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 491739bd674d493195daacf9afe48cb1: No bootstrap required, opened a new log
I20260619 17:02:46.963587 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 10686
I20260619 17:02:46.963696 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 17:02:46.964025 21406 external_mini_cluster.cc:1428] Setting key 6dda2c98d76a256d161103d649a7d099
I20260619 17:02:46.964010 10754 raft_consensus.cc:348] T 00000000000000000000000000000000 P 491739bd674d493195daacf9afe48cb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "491739bd674d493195daacf9afe48cb1" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 32901 } }
I20260619 17:02:46.964250 10754 raft_consensus.cc:374] T 00000000000000000000000000000000 P 491739bd674d493195daacf9afe48cb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:46.964298 10754 raft_consensus.cc:729] T 00000000000000000000000000000000 P 491739bd674d493195daacf9afe48cb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 491739bd674d493195daacf9afe48cb1, State: Initialized, Role: FOLLOWER
I20260619 17:02:46.964504 10754 consensus_queue.cc:260] T 00000000000000000000000000000000 P 491739bd674d493195daacf9afe48cb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "491739bd674d493195daacf9afe48cb1" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 32901 } }
I20260619 17:02:46.964643 10754 raft_consensus.cc:388] T 00000000000000000000000000000000 P 491739bd674d493195daacf9afe48cb1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 17:02:46.964694 10754 raft_consensus.cc:482] T 00000000000000000000000000000000 P 491739bd674d493195daacf9afe48cb1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 17:02:46.964752 10754 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 491739bd674d493195daacf9afe48cb1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:46.966053 10754 raft_consensus.cc:504] T 00000000000000000000000000000000 P 491739bd674d493195daacf9afe48cb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "491739bd674d493195daacf9afe48cb1" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 32901 } }
I20260619 17:02:46.966200 10754 leader_election.cc:302] T 00000000000000000000000000000000 P 491739bd674d493195daacf9afe48cb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 491739bd674d493195daacf9afe48cb1; no voters:
I20260619 17:02:46.966382 10754 leader_election.cc:288] T 00000000000000000000000000000000 P 491739bd674d493195daacf9afe48cb1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 17:02:46.966403 10759 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 491739bd674d493195daacf9afe48cb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:02:46.966674 10759 raft_consensus.cc:686] T 00000000000000000000000000000000 P 491739bd674d493195daacf9afe48cb1 [term 1 LEADER]: Becoming Leader. State: Replica: 491739bd674d493195daacf9afe48cb1, State: Running, Role: LEADER
I20260619 17:02:46.966801 10758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:46.964739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46517 (local address 127.20.231.190:32901)
0619 17:02:46.964974 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:46.964985 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:46.965022 (+ 37us) server_negotiation.cc:407] Connection header received
0619 17:02:46.965155 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:46.965162 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:46.965193 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:46.965374 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:46.966168 (+ 794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:46.966172 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:46.966172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:46.966177 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:46.966232 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:46.966248 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:46.966323 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:46.966467 (+ 144us) server_negotiation.cc:299] Negotiation successful
0619 17:02:46.966524 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":126,"threads_started":1}
I20260619 17:02:46.966821 10754 sys_catalog.cc:564] T 00000000000000000000000000000000 P 491739bd674d493195daacf9afe48cb1 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:02:46.967283 10759 consensus_queue.cc:237] T 00000000000000000000000000000000 P 491739bd674d493195daacf9afe48cb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "491739bd674d493195daacf9afe48cb1" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 32901 } }
I20260619 17:02:46.968070 21406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 17:02:46.968659 10760 sys_catalog.cc:455] T 00000000000000000000000000000000 P 491739bd674d493195daacf9afe48cb1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "491739bd674d493195daacf9afe48cb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "491739bd674d493195daacf9afe48cb1" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 32901 } } }
I20260619 17:02:46.969004 10760 sys_catalog.cc:458] T 00000000000000000000000000000000 P 491739bd674d493195daacf9afe48cb1 [sys.catalog]: This master's current role is: LEADER
W20260619 17:02:46.969172 10774 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 491739bd674d493195daacf9afe48cb1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 17:02:46.969241 10774 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 17:02:46.969350 10776 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 17:02:46.969616 10776 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 17:02:46.969594 10761 sys_catalog.cc:455] T 00000000000000000000000000000000 P 491739bd674d493195daacf9afe48cb1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 491739bd674d493195daacf9afe48cb1. Latest consensus state: current_term: 1 leader_uuid: "491739bd674d493195daacf9afe48cb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "491739bd674d493195daacf9afe48cb1" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 32901 } } }
I20260619 17:02:46.969717 10761 sys_catalog.cc:458] T 00000000000000000000000000000000 P 491739bd674d493195daacf9afe48cb1 [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:46.972079 10776 catalog_manager.cc:1347] Generated new cluster ID: e05125bb76af4465af6590d627b76dcb
I20260619 17:02:46.972204 10776 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 17:02:46.973825 10758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:46.971268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39161 (local address 127.20.231.190:32901)
0619 17:02:46.971310 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:46.971312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:46.972559 (+ 1247us) server_negotiation.cc:407] Connection header received
0619 17:02:46.972675 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:46.972676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:46.972683 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:46.972718 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:46.973516 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:46.973517 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:46.973518 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:46.973523 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:46.973541 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:46.973546 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:46.973580 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:46.973704 (+ 124us) server_negotiation.cc:299] Negotiation successful
0619 17:02:46.973723 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 17:02:46.993269 10776 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 17:02:46.993516 10776 catalog_manager.cc:1504] Loading token signing keys...
I20260619 17:02:46.998114 10776 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 491739bd674d493195daacf9afe48cb1: Generated new TSK 0
I20260619 17:02:46.998365 10776 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 17:02:47.004350 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 10686
2026-06-19T17:02:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1430 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 17:02:47 dist-test-slave-vnbj krb5kdc[10785](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 17:02:47 dist-test-slave-vnbj krb5kdc[10785](info): set up 2 sockets
Jun 19 17:02:47 dist-test-slave-vnbj krb5kdc[10785](info): commencing operation
krb5kdc: starting...
W20260619 17:02:48.079617 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.053s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 17:02:48 dist-test-slave-vnbj krb5kdc[10785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T17:02:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T17:02:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.231.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
I20260619 17:02:48.223425 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:32905
--webserver_interface=127.20.231.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:46811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:32905
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:02:48.318958 10798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:48.319262 10798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:48.319311 10798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:48.320638 10798 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:02:48.320730 10798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:48.320755 10798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:48.320786 10798 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:02:48.320843 10798 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:02:48.322604 10798 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:46811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:32905
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:32905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:48.322968 10798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:48.323259 10798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 17:02:48.328647 10798 server_base.cc:1034] running on GCE node
W20260619 17:02:48.328625 10803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:48.328625 10806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:48.328615 10804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:48.329249 10798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:48.329618 10798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:48.330966 10798 hybrid_clock.cc:648] HybridClock initialized: now 1781888568330933 us; error 67 us; skew 500 ppm
Jun 19 17:02:48 dist-test-slave-vnbj krb5kdc[10785](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888568, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:48.334054 10798 init.cc:377] Logged in from keytab as kudu/127.20.231.190@KRBTEST.COM (short username kudu)
I20260619 17:02:48.334641 10798 webserver.cc:460] Webserver started at http://127.20.231.190:38693/ using document root <none> and password file <none>
I20260619 17:02:48.334951 10798 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:48.335026 10798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:48.335191 10798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:48.336336 10798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "8c66ada988614b559db2d4fb2b0049a6"
format_stamp: "Formatted at 2026-06-19 17:02:48 on dist-test-slave-vnbj"
server_key: "41e66ddcabc43cb86dad6f342ea3c4c1"
server_key_iv: "71cbdf65c0643c5c9a9826fb14b3e75f"
server_key_version: "encryptionkey@0"
I20260619 17:02:48.336756 10798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "8c66ada988614b559db2d4fb2b0049a6"
format_stamp: "Formatted at 2026-06-19 17:02:48 on dist-test-slave-vnbj"
server_key: "41e66ddcabc43cb86dad6f342ea3c4c1"
server_key_iv: "71cbdf65c0643c5c9a9826fb14b3e75f"
server_key_version: "encryptionkey@0"
I20260619 17:02:48.338313 10798 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 17:02:48.339391 10813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:48.339596 10798 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 17:02:48.339710 10798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "8c66ada988614b559db2d4fb2b0049a6"
format_stamp: "Formatted at 2026-06-19 17:02:48 on dist-test-slave-vnbj"
server_key: "41e66ddcabc43cb86dad6f342ea3c4c1"
server_key_iv: "71cbdf65c0643c5c9a9826fb14b3e75f"
server_key_version: "encryptionkey@0"
I20260619 17:02:48.339915 10798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:48.357609 10798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:48.360368 10798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:48.360587 10798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:48.366067 10798 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:32905
I20260619 17:02:48.366103 10865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:32905 every 8 connection(s)
I20260619 17:02:48.366550 10798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 17:02:48.367667 10866 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:48.369912 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 10798
I20260619 17:02:48.370033 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 17:02:48.370237 21406 external_mini_cluster.cc:1428] Setting key 6bcc47f681ee16924787451e0489eeeb
I20260619 17:02:48.370968 10866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c66ada988614b559db2d4fb2b0049a6: Bootstrap starting.
I20260619 17:02:48.371824 10866 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c66ada988614b559db2d4fb2b0049a6: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:48.372185 10866 log.cc:824] T 00000000000000000000000000000000 P 8c66ada988614b559db2d4fb2b0049a6: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:48.373028 10866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c66ada988614b559db2d4fb2b0049a6: No bootstrap required, opened a new log
I20260619 17:02:48.375155 10866 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c66ada988614b559db2d4fb2b0049a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c66ada988614b559db2d4fb2b0049a6" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 32905 } }
I20260619 17:02:48.375347 10866 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c66ada988614b559db2d4fb2b0049a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:48.375425 10866 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c66ada988614b559db2d4fb2b0049a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c66ada988614b559db2d4fb2b0049a6, State: Initialized, Role: FOLLOWER
I20260619 17:02:48.375626 10866 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c66ada988614b559db2d4fb2b0049a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c66ada988614b559db2d4fb2b0049a6" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 32905 } }
I20260619 17:02:48.375728 10866 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c66ada988614b559db2d4fb2b0049a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 17:02:48.375773 10866 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c66ada988614b559db2d4fb2b0049a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 17:02:48.375828 10866 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c66ada988614b559db2d4fb2b0049a6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:48.376510 10866 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c66ada988614b559db2d4fb2b0049a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c66ada988614b559db2d4fb2b0049a6" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 32905 } }
I20260619 17:02:48.376685 10866 leader_election.cc:302] T 00000000000000000000000000000000 P 8c66ada988614b559db2d4fb2b0049a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c66ada988614b559db2d4fb2b0049a6; no voters:
I20260619 17:02:48.376905 10866 leader_election.cc:288] T 00000000000000000000000000000000 P 8c66ada988614b559db2d4fb2b0049a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 17:02:48.376958 10871 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c66ada988614b559db2d4fb2b0049a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:02:48.377180 10871 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c66ada988614b559db2d4fb2b0049a6 [term 1 LEADER]: Becoming Leader. State: Replica: 8c66ada988614b559db2d4fb2b0049a6, State: Running, Role: LEADER
I20260619 17:02:48.377228 10866 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c66ada988614b559db2d4fb2b0049a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:02:48.377398 10871 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c66ada988614b559db2d4fb2b0049a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c66ada988614b559db2d4fb2b0049a6" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 32905 } }
I20260619 17:02:48.377902 10873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c66ada988614b559db2d4fb2b0049a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c66ada988614b559db2d4fb2b0049a6. Latest consensus state: current_term: 1 leader_uuid: "8c66ada988614b559db2d4fb2b0049a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c66ada988614b559db2d4fb2b0049a6" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 32905 } } }
I20260619 17:02:48.378000 10873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c66ada988614b559db2d4fb2b0049a6 [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:48.378383 10872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c66ada988614b559db2d4fb2b0049a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c66ada988614b559db2d4fb2b0049a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c66ada988614b559db2d4fb2b0049a6" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 32905 } } }
I20260619 17:02:48.378453 10872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c66ada988614b559db2d4fb2b0049a6 [sys.catalog]: This master's current role is: LEADER
Jun 19 17:02:48 dist-test-slave-vnbj krb5kdc[10785](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
W20260619 17:02:48.379617 10886 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8c66ada988614b559db2d4fb2b0049a6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 17:02:48.379678 10886 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 17:02:48.379732 10880 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 17:02:48.379935 10880 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 17:02:48.383646 10869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:48.371048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49465 (local address 127.20.231.190:32905)
0619 17:02:48.371270 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:48.371281 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:48.371318 (+ 37us) server_negotiation.cc:407] Connection header received
0619 17:02:48.372086 (+ 768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:48.372097 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:48.372178 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:48.372347 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:48.373269 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:48.374308 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:48.375404 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:48.376164 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:48.376317 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:48.379558 (+ 3241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:48.379562 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:48.379563 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:48.379576 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:48.381497 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:48.382023 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:48.382027 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:48.382027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:48.382136 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:48.382452 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:48.382453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:48.382454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:48.382736 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:48.383081 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:48.383174 (+ 93us) server_negotiation.cc:299] Negotiation successful
0619 17:02:48.383392 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":106,"threads_started":1}
I20260619 17:02:48.384794 21406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 17:02:48.387368 10880 catalog_manager.cc:1347] Generated new cluster ID: ecf7ceef16be442882fecfa4d5e65ed2
I20260619 17:02:48.387449 10880 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 17:02:48.394289 10869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:48.386338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53045 (local address 127.20.231.190:32905)
0619 17:02:48.386385 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:48.386387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:48.386507 (+ 120us) server_negotiation.cc:407] Connection header received
0619 17:02:48.386559 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:48.386560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:48.386572 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:48.386647 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:48.387426 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:48.388208 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:48.389091 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:48.389415 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:48.389489 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:48.391284 (+ 1795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:48.391285 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:48.391286 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:48.391292 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:48.392940 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:48.393407 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:48.393408 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:48.393409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:48.393476 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:48.393712 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:48.393713 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:48.393713 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:48.393846 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:48.393955 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:48.394099 (+ 144us) server_negotiation.cc:299] Negotiation successful
0619 17:02:48.394185 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260619 17:02:48.412576 10880 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 17:02:48.413193 10880 catalog_manager.cc:1504] Loading token signing keys...
I20260619 17:02:48.417409 10880 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c66ada988614b559db2d4fb2b0049a6: Generated new TSK 0
I20260619 17:02:48.417637 10880 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 17:02:48.428251 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 10798
2026-06-19T17:02:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1420 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 17:02:48 dist-test-slave-vnbj krb5kdc[10897](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 17:02:48 dist-test-slave-vnbj krb5kdc[10897](info): set up 2 sockets
Jun 19 17:02:48 dist-test-slave-vnbj krb5kdc[10897](info): commencing operation
krb5kdc: starting...
W20260619 17:02:49.555392 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 17:02:49 dist-test-slave-vnbj krb5kdc[10897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888569, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T17:02:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T17:02:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.231.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
I20260619 17:02:49.708039 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:37869
--webserver_interface=127.20.231.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:43427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:37869
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:02:49.797966 10910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:49.798267 10910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:49.798300 10910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:49.800280 10910 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:02:49.800379 10910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:49.800398 10910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:49.800414 10910 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:02:49.800428 10910 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:02:49.803009 10910 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:43427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:37869
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:37869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:49.803335 10910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:49.803665 10910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:49.811174 10915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:49.811163 10918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:49.811633 10910 server_base.cc:1034] running on GCE node
W20260619 17:02:49.811163 10916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:49.812032 10910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:49.812945 10910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:49.814509 10910 hybrid_clock.cc:648] HybridClock initialized: now 1781888569814468 us; error 60 us; skew 500 ppm
Jun 19 17:02:49 dist-test-slave-vnbj krb5kdc[10897](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888569, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:49.834949 10910 init.cc:377] Logged in from keytab as kudu/127.20.231.190@KRBTEST.COM (short username kudu)
I20260619 17:02:49.835930 10910 webserver.cc:460] Webserver started at http://127.20.231.190:34221/ using document root <none> and password file <none>
I20260619 17:02:49.836553 10910 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:49.836688 10910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:49.836894 10910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:49.838323 10910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "0a894ef8aaf44eaf958bdd3d037e08fd"
format_stamp: "Formatted at 2026-06-19 17:02:49 on dist-test-slave-vnbj"
server_key: "ae4bbef0febab7488c20ca3f8ae47cb0"
server_key_iv: "8bf4eb860adb13d749f243f03a1e7dd7"
server_key_version: "encryptionkey@0"
I20260619 17:02:49.838758 10910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "0a894ef8aaf44eaf958bdd3d037e08fd"
format_stamp: "Formatted at 2026-06-19 17:02:49 on dist-test-slave-vnbj"
server_key: "ae4bbef0febab7488c20ca3f8ae47cb0"
server_key_iv: "8bf4eb860adb13d749f243f03a1e7dd7"
server_key_version: "encryptionkey@0"
I20260619 17:02:49.840688 10910 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260619 17:02:49.841952 10925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:49.842271 10910 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:49.842410 10910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "0a894ef8aaf44eaf958bdd3d037e08fd"
format_stamp: "Formatted at 2026-06-19 17:02:49 on dist-test-slave-vnbj"
server_key: "ae4bbef0febab7488c20ca3f8ae47cb0"
server_key_iv: "8bf4eb860adb13d749f243f03a1e7dd7"
server_key_version: "encryptionkey@0"
I20260619 17:02:49.842520 10910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:49.871008 10910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:49.873641 10910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:49.873838 10910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:49.878517 10910 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:37869
I20260619 17:02:49.878569 10977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:37869 every 8 connection(s)
I20260619 17:02:49.878993 10910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 17:02:49.880090 10978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:49.882731 10978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a894ef8aaf44eaf958bdd3d037e08fd: Bootstrap starting.
I20260619 17:02:49.883510 10978 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a894ef8aaf44eaf958bdd3d037e08fd: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:49.883803 10978 log.cc:824] T 00000000000000000000000000000000 P 0a894ef8aaf44eaf958bdd3d037e08fd: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:49.884672 10978 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a894ef8aaf44eaf958bdd3d037e08fd: No bootstrap required, opened a new log
I20260619 17:02:49.885572 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 10910
I20260619 17:02:49.885700 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 17:02:49.886042 21406 external_mini_cluster.cc:1428] Setting key 846194dad4909d62a60ae015a0ce569a
I20260619 17:02:49.887029 10978 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a894ef8aaf44eaf958bdd3d037e08fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a894ef8aaf44eaf958bdd3d037e08fd" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37869 } }
I20260619 17:02:49.887271 10978 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a894ef8aaf44eaf958bdd3d037e08fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:49.887315 10978 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a894ef8aaf44eaf958bdd3d037e08fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a894ef8aaf44eaf958bdd3d037e08fd, State: Initialized, Role: FOLLOWER
I20260619 17:02:49.887456 10978 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a894ef8aaf44eaf958bdd3d037e08fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a894ef8aaf44eaf958bdd3d037e08fd" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37869 } }
I20260619 17:02:49.887547 10978 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a894ef8aaf44eaf958bdd3d037e08fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 17:02:49.887585 10978 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a894ef8aaf44eaf958bdd3d037e08fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 17:02:49.887645 10978 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a894ef8aaf44eaf958bdd3d037e08fd [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:49.888592 10978 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a894ef8aaf44eaf958bdd3d037e08fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a894ef8aaf44eaf958bdd3d037e08fd" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37869 } }
I20260619 17:02:49.888721 10978 leader_election.cc:302] T 00000000000000000000000000000000 P 0a894ef8aaf44eaf958bdd3d037e08fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a894ef8aaf44eaf958bdd3d037e08fd; no voters:
I20260619 17:02:49.888922 10978 leader_election.cc:288] T 00000000000000000000000000000000 P 0a894ef8aaf44eaf958bdd3d037e08fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 17:02:49.888991 10983 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a894ef8aaf44eaf958bdd3d037e08fd [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:02:49.889214 10978 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a894ef8aaf44eaf958bdd3d037e08fd [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:02:49.889217 10983 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a894ef8aaf44eaf958bdd3d037e08fd [term 1 LEADER]: Becoming Leader. State: Replica: 0a894ef8aaf44eaf958bdd3d037e08fd, State: Running, Role: LEADER
I20260619 17:02:49.889413 10983 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a894ef8aaf44eaf958bdd3d037e08fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a894ef8aaf44eaf958bdd3d037e08fd" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37869 } }
I20260619 17:02:49.889874 10985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a894ef8aaf44eaf958bdd3d037e08fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a894ef8aaf44eaf958bdd3d037e08fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a894ef8aaf44eaf958bdd3d037e08fd" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37869 } } }
I20260619 17:02:49.889844 10983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a894ef8aaf44eaf958bdd3d037e08fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a894ef8aaf44eaf958bdd3d037e08fd. Latest consensus state: current_term: 1 leader_uuid: "0a894ef8aaf44eaf958bdd3d037e08fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a894ef8aaf44eaf958bdd3d037e08fd" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37869 } } }
I20260619 17:02:49.890156 10983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a894ef8aaf44eaf958bdd3d037e08fd [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:49.890383 10985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a894ef8aaf44eaf958bdd3d037e08fd [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:49.890681 10992 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 17:02:49.891534 10992 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 17:02:49.893709 10992 catalog_manager.cc:1347] Generated new cluster ID: 5411c0e84670401bbd799086666b852b
I20260619 17:02:49.893784 10992 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 17:02:49.894232 10982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:49.886837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56873 (local address 127.20.231.190:37869)
0619 17:02:49.887173 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:49.887183 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:49.887221 (+ 38us) server_negotiation.cc:407] Connection header received
0619 17:02:49.887388 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:49.887397 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:49.887463 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:49.887659 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:49.888106 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:49.889843 (+ 1737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:49.891483 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:49.892219 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:49.892391 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:49.893319 (+ 928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:49.893323 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:49.893324 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:49.893569 (+ 245us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:49.893627 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:49.893646 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:49.893767 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:49.893909 (+ 142us) server_negotiation.cc:299] Negotiation successful
0619 17:02:49.893991 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":140,"threads_started":1}
I20260619 17:02:49.895365 21406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 17:02:49.902998 10982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:49.897420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40403 (local address 127.20.231.190:37869)
0619 17:02:49.898957 (+ 1537us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:49.898959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:49.898970 (+ 11us) server_negotiation.cc:407] Connection header received
0619 17:02:49.899007 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:49.899008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:49.899022 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:49.899101 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:49.899323 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:49.900144 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:49.901391 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:49.901736 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:49.901816 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:49.902664 (+ 848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:49.902666 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:49.902666 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:49.902672 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:49.902689 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:49.902694 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:49.902749 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:49.902837 (+ 88us) server_negotiation.cc:299] Negotiation successful
0619 17:02:49.902899 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1505}
I20260619 17:02:49.922505 10992 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 17:02:49.923401 10992 catalog_manager.cc:1504] Loading token signing keys...
I20260619 17:02:49.933007 10992 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a894ef8aaf44eaf958bdd3d037e08fd: Generated new TSK 0
I20260619 17:02:49.933264 10992 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 17:02:49.942116 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 10910
2026-06-19T17:02:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1515 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 17:02:50 dist-test-slave-vnbj krb5kdc[11009](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 17:02:50 dist-test-slave-vnbj krb5kdc[11009](info): set up 2 sockets
Jun 19 17:02:50 dist-test-slave-vnbj krb5kdc[11009](info): commencing operation
krb5kdc: starting...
W20260619 17:02:51.087443 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 17:02:51 dist-test-slave-vnbj krb5kdc[11009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T17:02:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T17:02:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.231.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
I20260619 17:02:51.235328 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:39765
--webserver_interface=127.20.231.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:45419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:39765
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:02:51.332929 11022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:51.333178 11022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:51.333204 11022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:51.334434 11022 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:02:51.334491 11022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:51.334507 11022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:51.334520 11022 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:02:51.334532 11022 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:02:51.336186 11022 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:45419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:39765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:39765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:51.336429 11022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:51.336632 11022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:51.342171 11027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:51.342201 11028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:51.342171 11030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:51.342258 11022 server_base.cc:1034] running on GCE node
I20260619 17:02:51.342695 11022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:51.343088 11022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:51.344267 11022 hybrid_clock.cc:648] HybridClock initialized: now 1781888571344244 us; error 40 us; skew 500 ppm
Jun 19 17:02:51 dist-test-slave-vnbj krb5kdc[11009](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888571, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:51.347122 11022 init.cc:377] Logged in from keytab as kudu/127.20.231.190@KRBTEST.COM (short username kudu)
I20260619 17:02:51.347617 11022 webserver.cc:460] Webserver started at http://127.20.231.190:37267/ using document root <none> and password file <none>
I20260619 17:02:51.347862 11022 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:51.347936 11022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:51.348106 11022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:51.349248 11022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "a59aacd3273e4e1bb8f3bb2aadb281eb"
format_stamp: "Formatted at 2026-06-19 17:02:51 on dist-test-slave-vnbj"
server_key: "b56a9a9d6ef06c879b174e8026d3637f"
server_key_iv: "3c5d5c38208db764fca7de6b1670af05"
server_key_version: "encryptionkey@0"
I20260619 17:02:51.349684 11022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "a59aacd3273e4e1bb8f3bb2aadb281eb"
format_stamp: "Formatted at 2026-06-19 17:02:51 on dist-test-slave-vnbj"
server_key: "b56a9a9d6ef06c879b174e8026d3637f"
server_key_iv: "3c5d5c38208db764fca7de6b1670af05"
server_key_version: "encryptionkey@0"
I20260619 17:02:51.351188 11022 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260619 17:02:51.351967 11037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:51.352139 11022 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260619 17:02:51.352249 11022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "a59aacd3273e4e1bb8f3bb2aadb281eb"
format_stamp: "Formatted at 2026-06-19 17:02:51 on dist-test-slave-vnbj"
server_key: "b56a9a9d6ef06c879b174e8026d3637f"
server_key_iv: "3c5d5c38208db764fca7de6b1670af05"
server_key_version: "encryptionkey@0"
I20260619 17:02:51.352326 11022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:51.356046 11022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:51.358404 11022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:51.358600 11022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:51.363003 11022 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:39765
I20260619 17:02:51.363034 11089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:39765 every 8 connection(s)
I20260619 17:02:51.363422 11022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 17:02:51.364436 11090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:51.367055 11090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a59aacd3273e4e1bb8f3bb2aadb281eb: Bootstrap starting.
I20260619 17:02:51.367645 11090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a59aacd3273e4e1bb8f3bb2aadb281eb: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:51.367890 11090 log.cc:824] T 00000000000000000000000000000000 P a59aacd3273e4e1bb8f3bb2aadb281eb: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:51.368712 11090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a59aacd3273e4e1bb8f3bb2aadb281eb: No bootstrap required, opened a new log
I20260619 17:02:51.370380 11090 raft_consensus.cc:348] T 00000000000000000000000000000000 P a59aacd3273e4e1bb8f3bb2aadb281eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a59aacd3273e4e1bb8f3bb2aadb281eb" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 39765 } }
I20260619 17:02:51.370553 11090 raft_consensus.cc:374] T 00000000000000000000000000000000 P a59aacd3273e4e1bb8f3bb2aadb281eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 17:02:51.370579 11090 raft_consensus.cc:729] T 00000000000000000000000000000000 P a59aacd3273e4e1bb8f3bb2aadb281eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a59aacd3273e4e1bb8f3bb2aadb281eb, State: Initialized, Role: FOLLOWER
I20260619 17:02:51.370741 11090 consensus_queue.cc:260] T 00000000000000000000000000000000 P a59aacd3273e4e1bb8f3bb2aadb281eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a59aacd3273e4e1bb8f3bb2aadb281eb" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 39765 } }
I20260619 17:02:51.370829 11090 raft_consensus.cc:388] T 00000000000000000000000000000000 P a59aacd3273e4e1bb8f3bb2aadb281eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 17:02:51.370893 11090 raft_consensus.cc:482] T 00000000000000000000000000000000 P a59aacd3273e4e1bb8f3bb2aadb281eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 17:02:51.370999 11090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a59aacd3273e4e1bb8f3bb2aadb281eb [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:51.371531 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 11022
I20260619 17:02:51.371670 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 17:02:51.371867 21406 external_mini_cluster.cc:1428] Setting key 9f40b0b744da46adb13d64aa0cf94955
I20260619 17:02:51.372350 11090 raft_consensus.cc:504] T 00000000000000000000000000000000 P a59aacd3273e4e1bb8f3bb2aadb281eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a59aacd3273e4e1bb8f3bb2aadb281eb" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 39765 } }
I20260619 17:02:51.372476 11090 leader_election.cc:302] T 00000000000000000000000000000000 P a59aacd3273e4e1bb8f3bb2aadb281eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a59aacd3273e4e1bb8f3bb2aadb281eb; no voters:
I20260619 17:02:51.372686 11090 leader_election.cc:288] T 00000000000000000000000000000000 P a59aacd3273e4e1bb8f3bb2aadb281eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 17:02:51.372709 11095 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a59aacd3273e4e1bb8f3bb2aadb281eb [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:02:51.372881 11095 raft_consensus.cc:686] T 00000000000000000000000000000000 P a59aacd3273e4e1bb8f3bb2aadb281eb [term 1 LEADER]: Becoming Leader. State: Replica: a59aacd3273e4e1bb8f3bb2aadb281eb, State: Running, Role: LEADER
I20260619 17:02:51.373018 11090 sys_catalog.cc:564] T 00000000000000000000000000000000 P a59aacd3273e4e1bb8f3bb2aadb281eb [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:02:51.373059 11095 consensus_queue.cc:237] T 00000000000000000000000000000000 P a59aacd3273e4e1bb8f3bb2aadb281eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a59aacd3273e4e1bb8f3bb2aadb281eb" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 39765 } }
I20260619 17:02:51.373562 11095 sys_catalog.cc:455] T 00000000000000000000000000000000 P a59aacd3273e4e1bb8f3bb2aadb281eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader a59aacd3273e4e1bb8f3bb2aadb281eb. Latest consensus state: current_term: 1 leader_uuid: "a59aacd3273e4e1bb8f3bb2aadb281eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a59aacd3273e4e1bb8f3bb2aadb281eb" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 39765 } } }
I20260619 17:02:51.373824 11095 sys_catalog.cc:458] T 00000000000000000000000000000000 P a59aacd3273e4e1bb8f3bb2aadb281eb [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:51.373734 11096 sys_catalog.cc:455] T 00000000000000000000000000000000 P a59aacd3273e4e1bb8f3bb2aadb281eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a59aacd3273e4e1bb8f3bb2aadb281eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a59aacd3273e4e1bb8f3bb2aadb281eb" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 39765 } } }
I20260619 17:02:51.374257 11096 sys_catalog.cc:458] T 00000000000000000000000000000000 P a59aacd3273e4e1bb8f3bb2aadb281eb [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:51.374609 11094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:51.372463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59911 (local address 127.20.231.190:39765)
0619 17:02:51.372679 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:51.372690 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:51.372847 (+ 157us) server_negotiation.cc:407] Connection header received
0619 17:02:51.372992 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:51.373000 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:51.373043 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:51.373178 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:51.374032 (+ 854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:51.374036 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:51.374037 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:51.374042 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:51.374087 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:51.374103 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:51.374150 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:51.374322 (+ 172us) server_negotiation.cc:299] Negotiation successful
0619 17:02:51.374377 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":111,"threads_started":1}
W20260619 17:02:51.375037 11111 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a59aacd3273e4e1bb8f3bb2aadb281eb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 17:02:51.375092 11111 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 17:02:51.375144 11104 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 17:02:51.375327 11104 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 17:02:51.375921 21406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 17:02:51.377154 11104 catalog_manager.cc:1347] Generated new cluster ID: 8696903a63d54ce18c48c78ae38b3ee6
I20260619 17:02:51.377213 11104 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 17:02:51.384492 11094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:51.377567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42627 (local address 127.20.231.190:39765)
0619 17:02:51.377603 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:51.377605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:51.383113 (+ 5508us) server_negotiation.cc:407] Connection header received
0619 17:02:51.383197 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:51.383199 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:51.383208 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:51.383291 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:51.384118 (+ 827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:51.384120 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:51.384120 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 17:02:51.384128 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:51.384155 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 17:02:51.384164 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:51.384293 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:51.384358 (+ 65us) server_negotiation.cc:299] Negotiation successful
0619 17:02:51.384380 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 17:02:51.398308 11104 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 17:02:51.398540 11104 catalog_manager.cc:1504] Loading token signing keys...
I20260619 17:02:51.402127 11104 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a59aacd3273e4e1bb8f3bb2aadb281eb: Generated new TSK 0
I20260619 17:02:51.402359 11104 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 17:02:51.409188 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 11022
2026-06-19T17:02:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1465 ms)
[----------] 18 tests from AuthTokenIssuingTest (25499 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 17:02:51 dist-test-slave-vnbj krb5kdc[11121](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 17:02:51 dist-test-slave-vnbj krb5kdc[11121](info): set up 2 sockets
Jun 19 17:02:51 dist-test-slave-vnbj krb5kdc[11121](info): commencing operation
krb5kdc: starting...
W20260619 17:02:52.463248 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 17:02:52 dist-test-slave-vnbj krb5kdc[11121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T17:02:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T17:02:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.231.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
I20260619 17:02:52.603353 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:37111
--webserver_interface=127.20.231.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:36587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:37111,127.20.231.189:38845,127.20.231.188:45237
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:02:52.688997 11134 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 17:02:52.689247 11134 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:52.689266 11134 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:52.689291 11134 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:52.690671 11134 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:02:52.690725 11134 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:52.690740 11134 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:52.690754 11134 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:02:52.690766 11134 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:02:52.692438 11134 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:36587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:37111,127.20.231.189:38845,127.20.231.188:45237
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:37111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11134
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:52.692729 11134 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:52.692996 11134 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 17:02:52.698469 11134 server_base.cc:1034] running on GCE node
W20260619 17:02:52.698372 11140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:52.698415 11139 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:52.698370 11142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:52.698997 11134 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:52.699369 11134 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:52.700594 11134 hybrid_clock.cc:648] HybridClock initialized: now 1781888572700554 us; error 42 us; skew 500 ppm
Jun 19 17:02:52 dist-test-slave-vnbj krb5kdc[11121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888572, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:52.703627 11134 init.cc:377] Logged in from keytab as kudu/127.20.231.190@KRBTEST.COM (short username kudu)
I20260619 17:02:52.704177 11134 webserver.cc:460] Webserver started at http://127.20.231.190:35899/ using document root <none> and password file <none>
I20260619 17:02:52.704473 11134 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:52.704550 11134 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:52.704704 11134 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:52.705864 11134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "28f3d6f2644f47089feb42fb2517308b"
format_stamp: "Formatted at 2026-06-19 17:02:52 on dist-test-slave-vnbj"
server_key: "ab1334888f63d06ce85f214ebd372645"
server_key_iv: "5a34f2ada5ad29eaa40aa65135df0209"
server_key_version: "encryptionkey@0"
I20260619 17:02:52.706259 11134 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "28f3d6f2644f47089feb42fb2517308b"
format_stamp: "Formatted at 2026-06-19 17:02:52 on dist-test-slave-vnbj"
server_key: "ab1334888f63d06ce85f214ebd372645"
server_key_iv: "5a34f2ada5ad29eaa40aa65135df0209"
server_key_version: "encryptionkey@0"
I20260619 17:02:52.707886 11134 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260619 17:02:52.708806 11149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:52.709034 11134 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:52.709151 11134 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "28f3d6f2644f47089feb42fb2517308b"
format_stamp: "Formatted at 2026-06-19 17:02:52 on dist-test-slave-vnbj"
server_key: "ab1334888f63d06ce85f214ebd372645"
server_key_iv: "5a34f2ada5ad29eaa40aa65135df0209"
server_key_version: "encryptionkey@0"
I20260619 17:02:52.709236 11134 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:52.737118 11134 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:52.739916 11134 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:52.740114 11134 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:52.744735 11134 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:37111
I20260619 17:02:52.744773 11201 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:37111 every 8 connection(s)
I20260619 17:02:52.745215 11134 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 17:02:52.746239 11202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:52.748214 11202 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37111 } has no permanent_uuid. Determining permanent_uuid...
I20260619 17:02:52.749076 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 11134
I20260619 17:02:52.749230 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 17:02:52.749486 21406 external_mini_cluster.cc:1428] Setting key 81391ea2a549fa46c2750b64971d0c6f
Jun 19 17:02:52 dist-test-slave-vnbj krb5kdc[11121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888572, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.231.189@KRBTEST.COM; defaulting to no policy
I20260619 17:02:52.762094 11204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:52.749225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:37111 (local address 127.0.0.1:46998)
0619 17:02:52.749511 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:52.749577 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:52.750094 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:52.750475 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:52.750478 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:52.750745 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:52.751056 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:52.751063 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:52.752267 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:52.752269 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:52.753961 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:52.753967 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:52.755159 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:52.755161 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:52.755298 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:52.756062 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:02:52.756076 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:52.758047 (+ 1971us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:52.760174 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:52.760181 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:52.760184 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:52.760505 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:52.760782 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:52.760782 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:52.760783 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:52.760906 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:52.761332 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:52.761337 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:52.761447 (+ 110us) client_negotiation.cc:769] Sending connection context
0619 17:02:52.761595 (+ 148us) client_negotiation.cc:240] Negotiation successful
0619 17:02:52.761867 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":102,"thread_start_us":82,"threads_started":1}
I20260619 17:02:52.762092 11205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:52.749269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46998 (local address 127.20.231.190:37111)
0619 17:02:52.749500 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:52.749511 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:52.749689 (+ 178us) server_negotiation.cc:407] Connection header received
0619 17:02:52.750311 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:52.750314 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:52.750339 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:52.750450 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:52.751198 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:52.752155 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:52.754168 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:52.754805 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:52.755172 (+ 367us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:52.758180 (+ 3008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:52.758191 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:52.758192 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:52.758204 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:52.760010 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:52.760612 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:52.760616 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:52.760616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:52.760709 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:52.760978 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:52.760979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:52.760979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:52.761236 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:52.761353 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:52.761630 (+ 277us) server_negotiation.cc:299] Negotiation successful
0619 17:02:52.761820 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":177,"thread_start_us":88,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.189@KRBTEST.COM" created.
I20260619 17:02:52.763126 11202 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38845 } has no permanent_uuid. Determining permanent_uuid...
I20260619 17:02:52.763537 11204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:52.763406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.189:38845 (local address 127.0.0.1:49640)
0619 17:02:52.763457 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:52.763503 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.189:38845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260619 17:02:52.763705 11151 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.231.189:38845: connect: Connection refused (error 111)
I20260619 17:02:52.764200 11204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:52.764094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.189:38845 (local address 127.0.0.1:49656)
0619 17:02:52.764146 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:52.764173 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.189:38845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 17:02:52.764420 11202 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.231.189:38845: Network error: Client connection negotiation failed: client connection to 127.20.231.189:38845: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.189.keytab.
Entry for principal kudu/127.20.231.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.189.keytab.
I20260619 17:02:52.784392 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.189:38845
--webserver_interface=127.20.231.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.189.keytab
--principal=kudu/127.20.231.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:36587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:37111,127.20.231.189:38845,127.20.231.188:45237
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 17:02:52.813691 11202 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38845 } attempt: 1
I20260619 17:02:52.814261 11204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:52.814118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.189:38845 (local address 127.0.0.1:49672)
0619 17:02:52.814188 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:52.814231 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.189:38845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260619 17:02:52.814730 11204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:52.814635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.189:38845 (local address 127.0.0.1:49682)
0619 17:02:52.814684 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:52.814712 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.189:38845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 17:02:52.814883 11202 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.231.189:38845: Network error: Client connection negotiation failed: client connection to 127.20.231.189:38845: connect: Connection refused (error 111)
W20260619 17:02:52.877912 11209 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 17:02:52.878186 11209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:52.878206 11209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:52.878232 11209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:52.879767 11209 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:02:52.879846 11209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:52.879863 11209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:52.879877 11209 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:02:52.879891 11209 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:02:52.881784 11209 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:36587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:37111,127.20.231.189:38845,127.20.231.188:45237
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.189.keytab
--principal=kudu/127.20.231.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.189:38845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.231.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:52.882102 11209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:52.882376 11209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 17:02:52.883078 11202 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38845 } attempt: 2
I20260619 17:02:52.883705 11204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:52.883475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.189:38845 (local address 127.0.0.1:49684)
0619 17:02:52.883566 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:52.883660 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.189:38845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260619 17:02:52.884188 11204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:52.884063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.189:38845 (local address 127.0.0.1:49692)
0619 17:02:52.884121 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:52.884157 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.189:38845: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260619 17:02:52.884335 11202 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.231.189:38845: Network error: Client connection negotiation failed: client connection to 127.20.231.189:38845: connect: Connection refused (error 111)
W20260619 17:02:52.888633 11217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:52.888645 11215 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:52.888656 11214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:52.889008 11209 server_base.cc:1034] running on GCE node
I20260619 17:02:52.889283 11209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:52.889634 11209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:52.890949 11209 hybrid_clock.cc:648] HybridClock initialized: now 1781888572890897 us; error 69 us; skew 500 ppm
Jun 19 17:02:52 dist-test-slave-vnbj krb5kdc[11121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888572, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:52.893962 11209 init.cc:377] Logged in from keytab as kudu/127.20.231.189@KRBTEST.COM (short username kudu)
I20260619 17:02:52.894550 11209 webserver.cc:460] Webserver started at http://127.20.231.189:39181/ using document root <none> and password file <none>
I20260619 17:02:52.894812 11209 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:52.894918 11209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:52.895068 11209 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:52.896245 11209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-1/data/instance:
uuid: "60e035391ed04b9794d9d51903438029"
format_stamp: "Formatted at 2026-06-19 17:02:52 on dist-test-slave-vnbj"
server_key: "52f53abadc6c4f9ded6499018fb08125"
server_key_iv: "e6de67f8c57968ea9130b3c766c89fab"
server_key_version: "encryptionkey@0"
I20260619 17:02:52.896656 11209 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-1/wal/instance:
uuid: "60e035391ed04b9794d9d51903438029"
format_stamp: "Formatted at 2026-06-19 17:02:52 on dist-test-slave-vnbj"
server_key: "52f53abadc6c4f9ded6499018fb08125"
server_key_iv: "e6de67f8c57968ea9130b3c766c89fab"
server_key_version: "encryptionkey@0"
I20260619 17:02:52.898391 11209 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260619 17:02:52.899536 11224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:52.899739 11209 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260619 17:02:52.899928 11209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-1/wal
uuid: "60e035391ed04b9794d9d51903438029"
format_stamp: "Formatted at 2026-06-19 17:02:52 on dist-test-slave-vnbj"
server_key: "52f53abadc6c4f9ded6499018fb08125"
server_key_iv: "e6de67f8c57968ea9130b3c766c89fab"
server_key_version: "encryptionkey@0"
I20260619 17:02:52.900034 11209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:52.937521 11209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:52.940485 11209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:52.940721 11209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:52.945537 11209 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.189:38845
I20260619 17:02:52.945694 11276 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.189:38845 every 8 connection(s)
I20260619 17:02:52.946043 11209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-1/data/info.pb
I20260619 17:02:52.947116 11277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:52.949077 11277 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37111 } has no permanent_uuid. Determining permanent_uuid...
I20260619 17:02:52.951424 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 11209
I20260619 17:02:52.951536 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-1/wal/instance
I20260619 17:02:52.951747 21406 external_mini_cluster.cc:1428] Setting key 78df1090f64665b7c74eb32ba59aab0f
Jun 19 17:02:52 dist-test-slave-vnbj krb5kdc[11121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888572, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.189@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.231.188@KRBTEST.COM; defaulting to no policy
I20260619 17:02:52.961475 11205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:52.950034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47014 (local address 127.20.231.190:37111)
0619 17:02:52.950115 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:52.950117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:52.950344 (+ 227us) server_negotiation.cc:407] Connection header received
0619 17:02:52.950978 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:52.950979 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:52.950993 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:52.951061 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:52.952298 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:52.952819 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:52.954068 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:52.954323 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:52.954392 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:52.957706 (+ 3314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:52.957709 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:52.957709 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:52.957720 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:52.959585 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:52.960257 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:52.960259 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:52.960259 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:52.960334 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:52.960696 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:52.960697 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:52.960697 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:52.960919 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:52.960962 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:52.961298 (+ 336us) server_negotiation.cc:299] Negotiation successful
0619 17:02:52.961401 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 17:02:52.961690 11279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:52.949934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:37111 (local address 127.0.0.1:47014)
0619 17:02:52.950147 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:52.950210 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:52.950724 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:52.951231 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:52.951237 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:52.951671 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:52.952169 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:52.952177 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:52.953056 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:52.953057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:52.953947 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:52.953950 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:52.954425 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:52.954426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:52.954589 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:52.955322 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:02:52.955333 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:52.957518 (+ 2185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:52.959741 (+ 2223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:52.959752 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:52.959755 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:52.960153 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:52.960411 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:52.960412 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:52.960412 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:52.960590 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:52.961007 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:52.961013 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:52.961153 (+ 140us) client_negotiation.cc:769] Sending connection context
0619 17:02:52.961239 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 17:02:52.961386 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":128,"threads_started":1}
I20260619 17:02:52.962343 11277 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38845 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.188@KRBTEST.COM" created.
Jun 19 17:02:52 dist-test-slave-vnbj krb5kdc[11121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888572, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.189@KRBTEST.COM for kudu/127.20.231.189@KRBTEST.COM
I20260619 17:02:52.971853 11279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:52.962734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.189:38845 (local address 127.0.0.1:49698)
0619 17:02:52.962928 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:52.962938 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:52.962974 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:52.963159 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:52.963159 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:52.963339 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:52.963394 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:52.963396 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:52.964550 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:52.964551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:52.965404 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:52.965408 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:52.966348 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:52.966350 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:52.966428 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:52.967017 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:02:52.967024 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:52.968561 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:52.970635 (+ 2074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:52.970636 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:52.970637 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:52.970814 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:52.971093 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:52.971094 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:52.971094 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:52.971153 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:52.971509 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:52.971510 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:52.971551 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 17:02:52.971609 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 17:02:52.971726 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167}
I20260619 17:02:52.971923 11281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:52.962804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49698 (local address 127.20.231.189:38845)
0619 17:02:52.963018 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:52.963023 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:52.963039 (+ 16us) server_negotiation.cc:407] Connection header received
0619 17:02:52.963061 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:52.963063 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:52.963077 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:52.963147 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:52.963514 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:52.964431 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:52.965514 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:52.966186 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:52.966282 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:52.968673 (+ 2391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:52.968693 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:52.968694 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:52.968701 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:52.970519 (+ 1818us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:52.970959 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:52.970969 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:52.970970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:52.971023 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:52.971234 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:52.971235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:52.971235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:52.971426 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:52.971536 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:52.971660 (+ 124us) server_negotiation.cc:299] Negotiation successful
0619 17:02:52.971782 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":171,"thread_start_us":120,"threads_started":1}
I20260619 17:02:52.972518 11277 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.231.188" port: 45237 } has no permanent_uuid. Determining permanent_uuid...
I20260619 17:02:52.972896 11279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:52.972766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:45237 (local address 127.0.0.1:58630)
0619 17:02:52.972816 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:52.972864 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.188:45237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 17:02:52.973033 11226 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.231.188:45237: connect: Connection refused (error 111)
I20260619 17:02:52.973629 11279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:52.973529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:45237 (local address 127.0.0.1:58644)
0619 17:02:52.973579 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:52.973604 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.188:45237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260619 17:02:52.973822 11277 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.231.188:45237: Network error: Client connection negotiation failed: client connection to 127.20.231.188:45237: connect: Connection refused (error 111)
I20260619 17:02:52.975605 11202 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38845 } attempt: 3
Jun 19 17:02:52 dist-test-slave-vnbj krb5kdc[11121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888572, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for kudu/127.20.231.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.188.keytab.
Entry for principal kudu/127.20.231.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.188.keytab.
I20260619 17:02:52.984176 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.188:45237
--webserver_interface=127.20.231.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.188.keytab
--principal=kudu/127.20.231.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:36587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:37111,127.20.231.189:38845,127.20.231.188:45237
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 17:02:52.984555 11281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:52.976128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49700 (local address 127.20.231.189:38845)
0619 17:02:52.976169 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:52.976171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:52.976188 (+ 17us) server_negotiation.cc:407] Connection header received
0619 17:02:52.976215 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:52.976215 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:52.976226 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:52.976294 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:52.976668 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:52.977338 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:52.978292 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:52.978551 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:52.978618 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:52.981097 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:52.981099 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:52.981099 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:52.981108 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:52.982945 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:52.983517 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:52.983523 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:52.983524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:52.983586 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:52.983818 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:52.983819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:52.983820 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:52.984019 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:52.984110 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:52.984338 (+ 228us) server_negotiation.cc:299] Negotiation successful
0619 17:02:52.984453 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 17:02:52.984579 11204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:52.976009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.189:38845 (local address 127.0.0.1:49700)
0619 17:02:52.976110 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:52.976121 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:52.976167 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:52.976315 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:52.976316 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:52.976492 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:52.976566 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:52.976568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:52.977478 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:52.977479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:52.978202 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:52.978205 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:52.978625 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:52.978626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:52.978741 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:52.979251 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:02:52.979256 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:52.980996 (+ 1740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:52.983053 (+ 2057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:52.983055 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:52.983055 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:52.983289 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:52.983667 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:52.983668 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:52.983668 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:52.983743 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:52.984131 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:52.984132 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:52.984185 (+ 53us) client_negotiation.cc:769] Sending connection context
0619 17:02:52.984305 (+ 120us) client_negotiation.cc:240] Negotiation successful
0619 17:02:52.984473 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260619 17:02:52.985350 11202 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.231.188" port: 45237 } has no permanent_uuid. Determining permanent_uuid...
I20260619 17:02:52.985740 11204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:52.985620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:45237 (local address 127.0.0.1:58648)
0619 17:02:52.985675 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:52.985706 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.188:45237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 17:02:52.986130 11204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:52.986034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:45237 (local address 127.0.0.1:58656)
0619 17:02:52.986084 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:52.986110 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.188:45237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260619 17:02:52.986277 11202 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.231.188:45237: Network error: Client connection negotiation failed: client connection to 127.20.231.188:45237: connect: Connection refused (error 111)
I20260619 17:02:53.017449 11202 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.231.188" port: 45237 } attempt: 1
I20260619 17:02:53.018030 11204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:53.017873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:45237 (local address 127.0.0.1:58658)
0619 17:02:53.017942 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:53.017994 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.188:45237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 17:02:53.018489 11204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:53.018393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:45237 (local address 127.0.0.1:58664)
0619 17:02:53.018440 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:53.018469 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.188:45237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260619 17:02:53.018612 11202 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.231.188:45237: Network error: Client connection negotiation failed: client connection to 127.20.231.188:45237: connect: Connection refused (error 111)
I20260619 17:02:53.023077 11277 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.231.188" port: 45237 } attempt: 1
I20260619 17:02:53.023630 11279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:53.023464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:45237 (local address 127.0.0.1:58670)
0619 17:02:53.023538 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:53.023589 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.188:45237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260619 17:02:53.024070 11279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:53.023967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:45237 (local address 127.0.0.1:58676)
0619 17:02:53.024020 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:53.024049 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.188:45237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260619 17:02:53.024225 11277 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.231.188:45237: Network error: Client connection negotiation failed: client connection to 127.20.231.188:45237: connect: Connection refused (error 111)
W20260619 17:02:53.075102 11284 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 17:02:53.075376 11284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:53.075397 11284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:53.075421 11284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:53.076861 11284 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:02:53.076933 11284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:53.076949 11284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:53.076962 11284 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:02:53.076982 11284 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:02:53.078833 11284 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:36587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:37111,127.20.231.189:38845,127.20.231.188:45237
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.188.keytab
--principal=kudu/127.20.231.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.188:45237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.231.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:53.079165 11284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:53.079452 11284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 17:02:53.085150 11284 server_base.cc:1034] running on GCE node
W20260619 17:02:53.085150 11290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:53.085408 11292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:53.085425 11289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:53.085827 11284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:53.086166 11284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:53.087355 11284 hybrid_clock.cc:648] HybridClock initialized: now 1781888573087333 us; error 39 us; skew 500 ppm
Jun 19 17:02:53 dist-test-slave-vnbj krb5kdc[11121](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888573, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:53.090703 11284 init.cc:377] Logged in from keytab as kudu/127.20.231.188@KRBTEST.COM (short username kudu)
I20260619 17:02:53.091349 11284 webserver.cc:460] Webserver started at http://127.20.231.188:38701/ using document root <none> and password file <none>
I20260619 17:02:53.091605 11284 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:53.091662 11284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:53.091902 11284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:53.092463 11277 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.231.188" port: 45237 } attempt: 2
I20260619 17:02:53.093008 11279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:53.092871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:45237 (local address 127.0.0.1:58682)
0619 17:02:53.092937 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:53.092978 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.188:45237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260619 17:02:53.093331 11279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:53.093241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:45237 (local address 127.0.0.1:58692)
0619 17:02:53.093282 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:53.093306 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.188:45237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260619 17:02:53.093243 11284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-2/data/instance:
uuid: "4d9c9f08ec054f01b6d8472bb31c162d"
format_stamp: "Formatted at 2026-06-19 17:02:53 on dist-test-slave-vnbj"
server_key: "ec9fa641969b42bf6c6daffd8fa009cf"
server_key_iv: "e540585e706a57afe6b2b92d4c2386dd"
server_key_version: "encryptionkey@0"
W20260619 17:02:53.093442 11277 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.231.188:45237: Network error: Client connection negotiation failed: client connection to 127.20.231.188:45237: connect: Connection refused (error 111)
I20260619 17:02:53.093739 11284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-2/wal/instance:
uuid: "4d9c9f08ec054f01b6d8472bb31c162d"
format_stamp: "Formatted at 2026-06-19 17:02:53 on dist-test-slave-vnbj"
server_key: "ec9fa641969b42bf6c6daffd8fa009cf"
server_key_iv: "e540585e706a57afe6b2b92d4c2386dd"
server_key_version: "encryptionkey@0"
I20260619 17:02:53.093868 11202 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.231.188" port: 45237 } attempt: 2
I20260619 17:02:53.094270 11204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:53.094146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:45237 (local address 127.0.0.1:58702)
0619 17:02:53.094204 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:53.094234 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.188:45237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 17:02:53.094686 11204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:53.094618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:45237 (local address 127.0.0.1:58706)
0619 17:02:53.094646 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:53.094669 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.188:45237: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260619 17:02:53.094817 11202 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.231.188:45237: Network error: Client connection negotiation failed: client connection to 127.20.231.188:45237: connect: Connection refused (error 111)
I20260619 17:02:53.095613 11284 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260619 17:02:53.096657 11299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:53.096938 11284 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 17:02:53.097054 11284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-2/wal
uuid: "4d9c9f08ec054f01b6d8472bb31c162d"
format_stamp: "Formatted at 2026-06-19 17:02:53 on dist-test-slave-vnbj"
server_key: "ec9fa641969b42bf6c6daffd8fa009cf"
server_key_iv: "e540585e706a57afe6b2b92d4c2386dd"
server_key_version: "encryptionkey@0"
I20260619 17:02:53.097142 11284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:53.110724 11284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:53.113296 11284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:53.113518 11284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:53.118011 11284 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.188:45237
I20260619 17:02:53.118144 11351 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.188:45237 every 8 connection(s)
I20260619 17:02:53.118455 11284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-2/data/info.pb
I20260619 17:02:53.119592 11352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:53.120890 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 11284
I20260619 17:02:53.121004 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781888301345144-21406-0/minicluster-data/master-2/wal/instance
I20260619 17:02:53.121205 21406 external_mini_cluster.cc:1428] Setting key c6b58c6bbcb16895464785d7a58a23e5
I20260619 17:02:53.121680 11352 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37111 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 17:02:53 dist-test-slave-vnbj krb5kdc[11121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888573, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.188@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
Jun 19 17:02:53 dist-test-slave-vnbj krb5kdc[11121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.188@KRBTEST.COM
I20260619 17:02:53.132124 11205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:53.122625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47030 (local address 127.20.231.190:37111)
0619 17:02:53.122671 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:53.122673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:53.122983 (+ 310us) server_negotiation.cc:407] Connection header received
0619 17:02:53.123019 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:53.123020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:53.123030 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:53.123099 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:53.123723 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:53.124215 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:53.125138 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:53.125341 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:53.125417 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:53.128158 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:53.128160 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:53.128161 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:53.128169 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:53.129991 (+ 1822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:53.130717 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:53.130718 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:53.130718 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:53.130782 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:53.131229 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:53.131231 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:53.131231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:53.131421 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:53.131499 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:53.131815 (+ 316us) server_negotiation.cc:299] Negotiation successful
0619 17:02:53.132021 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 17:02:53.132282 11356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:53.122567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:37111 (local address 127.0.0.1:47030)
0619 17:02:53.122739 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:53.122783 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:53.122884 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:53.123125 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:53.123128 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:53.123444 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:53.123623 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:53.123627 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:53.124305 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:53.124306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:53.125046 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:53.125048 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:53.125438 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:53.125439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:53.125560 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:53.126132 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:02:53.126143 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:53.128044 (+ 1901us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:53.130096 (+ 2052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:53.130101 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:53.130104 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:53.130595 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:53.130994 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:53.130995 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:53.130995 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:53.131150 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:53.131498 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:53.131504 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:53.131653 (+ 149us) client_negotiation.cc:769] Sending connection context
0619 17:02:53.131773 (+ 120us) client_negotiation.cc:240] Negotiation successful
0619 17:02:53.131990 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":18,"thread_start_us":80,"threads_started":1}
I20260619 17:02:53.132287 11355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:53.121817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58714 (local address 127.20.231.188:45237)
0619 17:02:53.122018 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:53.122048 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:53.122088 (+ 40us) server_negotiation.cc:407] Connection header received
0619 17:02:53.122739 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:53.122764 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:53.122831 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:53.123098 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:53.123749 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:53.124601 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:53.125401 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:53.125711 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:53.125796 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:53.128589 (+ 2793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:53.128596 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:53.128597 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:53.128604 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:53.130620 (+ 2016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:53.131070 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:53.131072 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:53.131073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:53.131150 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:53.131398 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:53.131399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:53.131400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:53.131657 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:53.131801 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:53.131896 (+ 95us) server_negotiation.cc:299] Negotiation successful
0619 17:02:53.132043 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":141,"thread_start_us":105,"threads_started":1}
I20260619 17:02:53.132963 11352 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38845 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 17:02:53 dist-test-slave-vnbj krb5kdc[11121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888573, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.188@KRBTEST.COM for kudu/127.20.231.189@KRBTEST.COM
Jun 19 17:02:53 dist-test-slave-vnbj krb5kdc[11121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:02:53.142582 11356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:53.133271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.189:38845 (local address 127.0.0.1:49714)
0619 17:02:53.133397 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:53.133407 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:53.133454 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:53.133606 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:53.133607 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:53.133774 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:53.133843 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:53.133855 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:53.134986 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:53.134987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:53.135810 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:53.135814 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:53.136481 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:53.136482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:53.136568 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:53.137208 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:02:53.137214 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:53.138989 (+ 1775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:53.141466 (+ 2477us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:53.141467 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:53.141468 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:53.141651 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:53.141874 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:53.141875 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:53.141876 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:53.141939 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:53.142200 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:53.142201 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:53.142259 (+ 58us) client_negotiation.cc:769] Sending connection context
0619 17:02:53.142350 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 17:02:53.142469 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260619 17:02:53.142633 11281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:53.133328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49714 (local address 127.20.231.189:38845)
0619 17:02:53.133373 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:53.133376 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:53.133442 (+ 66us) server_negotiation.cc:407] Connection header received
0619 17:02:53.133511 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:53.133512 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:53.133523 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:53.133598 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:53.133982 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:53.134755 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:53.135921 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:53.136346 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:53.136423 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:53.139093 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:53.139095 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:53.139096 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:53.139106 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:53.141347 (+ 2241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:53.141730 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:53.141731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:53.141731 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:53.141797 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:53.142010 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:53.142011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:53.142012 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:53.142131 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:53.142207 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:53.142466 (+ 259us) server_negotiation.cc:299] Negotiation successful
0619 17:02:53.142540 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 17:02:53.142776 11205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:53.133313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47032 (local address 127.20.231.190:37111)
0619 17:02:53.133350 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:53.133352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:53.133363 (+ 11us) server_negotiation.cc:407] Connection header received
0619 17:02:53.133458 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:53.133459 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:53.133470 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:53.133534 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:53.134159 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:53.134982 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:53.136000 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:53.136285 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:53.136377 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:53.139611 (+ 3234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:53.139614 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:53.139614 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:53.139624 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:53.141137 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:53.141531 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:53.141532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:53.141533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:53.141595 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:53.141803 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:53.141804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:53.141805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:53.141931 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:53.142535 (+ 604us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:53.142569 (+ 34us) server_negotiation.cc:299] Negotiation successful
0619 17:02:53.142653 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260619 17:02:53.143018 11352 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.231.188" port: 45237 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 17:02:53 dist-test-slave-vnbj krb5kdc[11121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888573, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.188@KRBTEST.COM for kudu/127.20.231.188@KRBTEST.COM
Jun 19 17:02:53 dist-test-slave-vnbj krb5kdc[11121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.189@KRBTEST.COM
I20260619 17:02:53.151108 11355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:53.143374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58728 (local address 127.20.231.188:45237)
0619 17:02:53.143492 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:53.143494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:53.143500 (+ 6us) server_negotiation.cc:407] Connection header received
0619 17:02:53.143517 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:53.143518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:53.143528 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:53.143575 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:53.143829 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:53.144309 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:53.145003 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:53.145222 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:53.145319 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:53.147729 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:53.147731 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:53.147732 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:53.147740 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:53.149486 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:53.149990 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:53.149991 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:53.149992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:53.150054 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:53.150378 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:53.150379 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:53.150380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:53.150569 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:53.150618 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:53.150830 (+ 212us) server_negotiation.cc:299] Negotiation successful
0619 17:02:53.150959 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260619 17:02:53.151275 11281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:53.143433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49722 (local address 127.20.231.189:38845)
0619 17:02:53.143468 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:53.143470 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:53.143479 (+ 9us) server_negotiation.cc:407] Connection header received
0619 17:02:53.143541 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:53.143541 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:53.143549 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:53.143614 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:53.144088 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:53.144562 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:53.145245 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:53.145459 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:53.145547 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:53.147922 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:53.147923 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:53.147923 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:53.147930 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:53.149606 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:53.150235 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:53.150236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:53.150237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:53.150294 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:53.150537 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:53.150537 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:53.150538 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:53.150691 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:53.150958 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:53.151073 (+ 115us) server_negotiation.cc:299] Negotiation successful
0619 17:02:53.151177 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260619 17:02:53.151206 11356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:53.143321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:45237 (local address 127.0.0.1:58728)
0619 17:02:53.143375 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:53.143387 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:53.143427 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:53.143577 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:53.143578 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:53.143721 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:53.143755 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:53.143757 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:53.144399 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:53.144400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:53.144906 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:53.144907 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:53.145317 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:53.145318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:53.145367 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:53.145873 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:02:53.145878 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:53.147634 (+ 1756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:53.149590 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:53.149591 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:53.149592 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:53.149809 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:53.150169 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:53.150170 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:53.150171 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:53.150283 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:53.150637 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:53.150637 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:53.150691 (+ 54us) client_negotiation.cc:769] Sending connection context
0619 17:02:53.150767 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 17:02:53.151031 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260619 17:02:53.153952 21406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 17:02:53.154089 11352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d9c9f08ec054f01b6d8472bb31c162d: Bootstrap starting.
I20260619 17:02:53.155293 11352 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d9c9f08ec054f01b6d8472bb31c162d: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:53.155648 11352 log.cc:824] T 00000000000000000000000000000000 P 4d9c9f08ec054f01b6d8472bb31c162d: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:53.155164 11176 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "28f3d6f2644f47089feb42fb2517308b"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47032
I20260619 17:02:53.156757 11352 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d9c9f08ec054f01b6d8472bb31c162d: No bootstrap required, opened a new log
I20260619 17:02:53.158361 11352 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d9c9f08ec054f01b6d8472bb31c162d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28f3d6f2644f47089feb42fb2517308b" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37111 } } peers { permanent_uuid: "60e035391ed04b9794d9d51903438029" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38845 } } peers { permanent_uuid: "4d9c9f08ec054f01b6d8472bb31c162d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 45237 } }
I20260619 17:02:53.158537 11352 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d9c9f08ec054f01b6d8472bb31c162d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d9c9f08ec054f01b6d8472bb31c162d, State: Initialized, Role: FOLLOWER
I20260619 17:02:53.158677 11352 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d9c9f08ec054f01b6d8472bb31c162d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28f3d6f2644f47089feb42fb2517308b" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37111 } } peers { permanent_uuid: "60e035391ed04b9794d9d51903438029" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38845 } } peers { permanent_uuid: "4d9c9f08ec054f01b6d8472bb31c162d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 45237 } }
I20260619 17:02:53.159137 11352 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d9c9f08ec054f01b6d8472bb31c162d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:02:53.159134 11358 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d9c9f08ec054f01b6d8472bb31c162d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28f3d6f2644f47089feb42fb2517308b" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37111 } } peers { permanent_uuid: "60e035391ed04b9794d9d51903438029" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38845 } } peers { permanent_uuid: "4d9c9f08ec054f01b6d8472bb31c162d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 45237 } } }
I20260619 17:02:53.159317 11358 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d9c9f08ec054f01b6d8472bb31c162d [sys.catalog]: This master's current role is: FOLLOWER
W20260619 17:02:53.161307 11369 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4d9c9f08ec054f01b6d8472bb31c162d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 17:02:53.161391 11369 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 17:02:53.161875 11284 master_runner.cc:186] Error getting master registration for 127.20.231.190:37111: OK, instance_id { permanent_uuid: "28f3d6f2644f47089feb42fb2517308b" instance_seqno: 1781888572743414 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 17:02:53.164345 11284 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 17:02:53.184638 11277 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.231.188" port: 45237 } attempt: 3
Jun 19 17:02:53 dist-test-slave-vnbj krb5kdc[11121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888572, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.189@KRBTEST.COM for kudu/127.20.231.188@KRBTEST.COM
I20260619 17:02:53.193992 11279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:53.185170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:45237 (local address 127.0.0.1:58732)
0619 17:02:53.185243 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:53.185256 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:53.185364 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:53.185620 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:53.185621 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:53.185891 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:53.185951 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:53.185953 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:53.187014 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:53.187015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:53.187834 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:53.187836 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:53.188337 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:53.188338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:53.188407 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:53.188957 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:02:53.188963 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:53.190629 (+ 1666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:53.192663 (+ 2034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:53.192665 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:53.192665 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:53.192953 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:53.193188 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:53.193188 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:53.193189 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:53.193255 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:53.193612 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:53.193613 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:53.193682 (+ 69us) client_negotiation.cc:769] Sending connection context
0619 17:02:53.193752 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 17:02:53.193877 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260619 17:02:53.193959 11202 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.231.188" port: 45237 } attempt: 3
I20260619 17:02:53.194495 11355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:53.185253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58732 (local address 127.20.231.188:45237)
0619 17:02:53.185315 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:53.185318 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:53.185351 (+ 33us) server_negotiation.cc:407] Connection header received
0619 17:02:53.185435 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:53.185436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:53.185448 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:53.185526 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:53.186066 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:53.186909 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:53.187928 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:53.188243 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:53.188341 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:53.190738 (+ 2397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:53.190740 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:53.190741 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:53.190750 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:53.192546 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:53.193043 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:53.193044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:53.193044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:53.193113 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:53.193323 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:53.193324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:53.193324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:53.193542 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:53.193616 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:53.193792 (+ 176us) server_negotiation.cc:299] Negotiation successful
0619 17:02:53.193880 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 17:02:53.196221 11277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60e035391ed04b9794d9d51903438029: Bootstrap starting.
I20260619 17:02:53.196911 11277 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60e035391ed04b9794d9d51903438029: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:53.197229 11277 log.cc:824] T 00000000000000000000000000000000 P 60e035391ed04b9794d9d51903438029: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:53.198367 11277 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60e035391ed04b9794d9d51903438029: No bootstrap required, opened a new log
Jun 19 17:02:53 dist-test-slave-vnbj krb5kdc[11121](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888572, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for kudu/127.20.231.188@KRBTEST.COM
I20260619 17:02:53.200665 11277 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60e035391ed04b9794d9d51903438029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28f3d6f2644f47089feb42fb2517308b" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37111 } } peers { permanent_uuid: "60e035391ed04b9794d9d51903438029" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38845 } } peers { permanent_uuid: "4d9c9f08ec054f01b6d8472bb31c162d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 45237 } }
I20260619 17:02:53.200842 11277 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60e035391ed04b9794d9d51903438029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60e035391ed04b9794d9d51903438029, State: Initialized, Role: FOLLOWER
I20260619 17:02:53.200943 11277 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60e035391ed04b9794d9d51903438029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28f3d6f2644f47089feb42fb2517308b" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37111 } } peers { permanent_uuid: "60e035391ed04b9794d9d51903438029" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38845 } } peers { permanent_uuid: "4d9c9f08ec054f01b6d8472bb31c162d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 45237 } }
I20260619 17:02:53.201385 11277 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60e035391ed04b9794d9d51903438029 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:02:53.201305 11373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60e035391ed04b9794d9d51903438029 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28f3d6f2644f47089feb42fb2517308b" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37111 } } peers { permanent_uuid: "60e035391ed04b9794d9d51903438029" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38845 } } peers { permanent_uuid: "4d9c9f08ec054f01b6d8472bb31c162d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 45237 } } }
I20260619 17:02:53.201576 11373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60e035391ed04b9794d9d51903438029 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 17:02:53.204005 11204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:53.194238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:45237 (local address 127.0.0.1:58734)
0619 17:02:53.194308 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:53.194319 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:53.194357 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:53.194896 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:53.194898 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:53.195081 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:53.195136 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:53.195138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:53.196113 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:53.196114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:53.196949 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:53.196952 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:53.197482 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:53.197484 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:53.197557 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:53.198092 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:02:53.198098 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:53.199752 (+ 1654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:53.202587 (+ 2835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:53.202589 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:53.202590 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:53.202972 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:53.203253 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:53.203255 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:53.203255 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:53.203333 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:53.203667 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:53.203668 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:53.203726 (+ 58us) client_negotiation.cc:769] Sending connection context
0619 17:02:53.203789 (+ 63us) client_negotiation.cc:240] Negotiation successful
0619 17:02:53.203903 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260619 17:02:53.204066 11371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:53.194311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58734 (local address 127.20.231.188:45237)
0619 17:02:53.194531 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:53.194534 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:53.194650 (+ 116us) server_negotiation.cc:407] Connection header received
0619 17:02:53.194674 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:53.194675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:53.194687 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:53.194888 (+ 201us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:53.195229 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:53.196011 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:53.197051 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:53.197397 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:53.197466 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:53.199915 (+ 2449us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:53.199916 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:53.199917 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:53.199925 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:53.202434 (+ 2509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:53.203069 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:53.203071 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:53.203071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:53.203149 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:53.203414 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:53.203415 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:53.203416 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:53.203593 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:53.203660 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:53.203842 (+ 182us) server_negotiation.cc:299] Negotiation successful
0619 17:02:53.203945 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":120,"threads_started":1}
I20260619 17:02:53.205817 11202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28f3d6f2644f47089feb42fb2517308b: Bootstrap starting.
I20260619 17:02:53.203510 11209 master_runner.cc:186] Error getting master registration for 127.20.231.190:37111: OK, instance_id { permanent_uuid: "28f3d6f2644f47089feb42fb2517308b" instance_seqno: 1781888572743414 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 17:02:53.206578 11202 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28f3d6f2644f47089feb42fb2517308b: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:53.206900 11202 log.cc:824] T 00000000000000000000000000000000 P 28f3d6f2644f47089feb42fb2517308b: Log is configured to *not* fsync() on all Append() calls
W20260619 17:02:53.204396 11384 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 60e035391ed04b9794d9d51903438029: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 17:02:53.206982 11384 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 17:02:53.206633 11209 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 17:02:53.207794 11202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28f3d6f2644f47089feb42fb2517308b: No bootstrap required, opened a new log
I20260619 17:02:53.209184 11202 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28f3d6f2644f47089feb42fb2517308b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28f3d6f2644f47089feb42fb2517308b" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37111 } } peers { permanent_uuid: "60e035391ed04b9794d9d51903438029" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38845 } } peers { permanent_uuid: "4d9c9f08ec054f01b6d8472bb31c162d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 45237 } }
I20260619 17:02:53.209436 11202 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28f3d6f2644f47089feb42fb2517308b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28f3d6f2644f47089feb42fb2517308b, State: Initialized, Role: FOLLOWER
I20260619 17:02:53.209546 11202 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28f3d6f2644f47089feb42fb2517308b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28f3d6f2644f47089feb42fb2517308b" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37111 } } peers { permanent_uuid: "60e035391ed04b9794d9d51903438029" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38845 } } peers { permanent_uuid: "4d9c9f08ec054f01b6d8472bb31c162d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 45237 } }
I20260619 17:02:53.209777 11387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28f3d6f2644f47089feb42fb2517308b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28f3d6f2644f47089feb42fb2517308b" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37111 } } peers { permanent_uuid: "60e035391ed04b9794d9d51903438029" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38845 } } peers { permanent_uuid: "4d9c9f08ec054f01b6d8472bb31c162d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 45237 } } }
I20260619 17:02:53.209833 11202 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28f3d6f2644f47089feb42fb2517308b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:02:53.209885 11387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28f3d6f2644f47089feb42fb2517308b [sys.catalog]: This master's current role is: FOLLOWER
I20260619 17:02:53.209954 11176 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28f3d6f2644f47089feb42fb2517308b [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 17:02:53.210004 11176 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28f3d6f2644f47089feb42fb2517308b [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:53.210757 11176 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28f3d6f2644f47089feb42fb2517308b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28f3d6f2644f47089feb42fb2517308b" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37111 } } peers { permanent_uuid: "60e035391ed04b9794d9d51903438029" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38845 } } peers { permanent_uuid: "4d9c9f08ec054f01b6d8472bb31c162d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 45237 } }
I20260619 17:02:53.211047 11176 leader_election.cc:288] T 00000000000000000000000000000000 P 28f3d6f2644f47089feb42fb2517308b [CANDIDATE]: Term 1 election: Requested vote from peers 60e035391ed04b9794d9d51903438029 (127.20.231.189:38845), 4d9c9f08ec054f01b6d8472bb31c162d (127.20.231.188:45237)
I20260619 17:02:53.211404 11251 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "28f3d6f2644f47089feb42fb2517308b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "60e035391ed04b9794d9d51903438029"
I20260619 17:02:53.211539 11251 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60e035391ed04b9794d9d51903438029 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:53.212510 11251 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 60e035391ed04b9794d9d51903438029 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28f3d6f2644f47089feb42fb2517308b in term 1.
I20260619 17:02:53.212852 11151 leader_election.cc:302] T 00000000000000000000000000000000 P 28f3d6f2644f47089feb42fb2517308b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28f3d6f2644f47089feb42fb2517308b, 60e035391ed04b9794d9d51903438029; no voters:
I20260619 17:02:53.212970 11387 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28f3d6f2644f47089feb42fb2517308b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:02:53.212980 11326 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "28f3d6f2644f47089feb42fb2517308b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4d9c9f08ec054f01b6d8472bb31c162d"
I20260619 17:02:53.213107 11326 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d9c9f08ec054f01b6d8472bb31c162d [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:53.213163 11387 raft_consensus.cc:686] T 00000000000000000000000000000000 P 28f3d6f2644f47089feb42fb2517308b [term 1 LEADER]: Becoming Leader. State: Replica: 28f3d6f2644f47089feb42fb2517308b, State: Running, Role: LEADER
I20260619 17:02:53.213253 11387 consensus_queue.cc:237] T 00000000000000000000000000000000 P 28f3d6f2644f47089feb42fb2517308b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28f3d6f2644f47089feb42fb2517308b" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37111 } } peers { permanent_uuid: "60e035391ed04b9794d9d51903438029" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38845 } } peers { permanent_uuid: "4d9c9f08ec054f01b6d8472bb31c162d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 45237 } }
I20260619 17:02:53.214041 11326 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4d9c9f08ec054f01b6d8472bb31c162d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 28f3d6f2644f47089feb42fb2517308b in term 1.
I20260619 17:02:53.214056 11405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28f3d6f2644f47089feb42fb2517308b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28f3d6f2644f47089feb42fb2517308b. Latest consensus state: current_term: 1 leader_uuid: "28f3d6f2644f47089feb42fb2517308b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28f3d6f2644f47089feb42fb2517308b" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37111 } } peers { permanent_uuid: "60e035391ed04b9794d9d51903438029" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38845 } } peers { permanent_uuid: "4d9c9f08ec054f01b6d8472bb31c162d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 45237 } } }
I20260619 17:02:53.214138 11405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28f3d6f2644f47089feb42fb2517308b [sys.catalog]: This master's current role is: LEADER
W20260619 17:02:53.214787 11407 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 28f3d6f2644f47089feb42fb2517308b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 17:02:53.214877 11407 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 17:02:53.222062 11281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:53.212518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49724 (local address 127.20.231.189:38845)
0619 17:02:53.212562 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:53.212564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:53.213688 (+ 1124us) server_negotiation.cc:407] Connection header received
0619 17:02:53.214456 (+ 768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:53.214457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:53.214472 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:53.214539 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:53.215457 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:53.215961 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:53.216912 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:53.217197 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:53.217268 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:53.218749 (+ 1481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:53.218750 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:53.218751 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:53.218759 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:53.220686 (+ 1927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:53.221160 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:53.221162 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:53.221162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:53.221224 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:53.221440 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:53.221441 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:53.221442 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:53.221578 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:53.221659 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:53.221882 (+ 223us) server_negotiation.cc:299] Negotiation successful
0619 17:02:53.221963 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 17:02:53.222086 11205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:53.212303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47034 (local address 127.20.231.190:37111)
0619 17:02:53.212379 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:53.212382 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:53.213609 (+ 1227us) server_negotiation.cc:407] Connection header received
0619 17:02:53.214471 (+ 862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:53.214472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:53.214482 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:53.214513 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:53.215344 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:53.216042 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:53.217193 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:53.217495 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:53.217543 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:53.219031 (+ 1488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:53.219033 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:53.219033 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:53.219040 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:53.220833 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:53.221273 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:53.221274 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:53.221275 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:53.221330 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:53.221528 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:53.221529 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:53.221529 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:53.221651 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:53.221719 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:53.221917 (+ 198us) server_negotiation.cc:299] Negotiation successful
0619 17:02:53.221994 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 17:02:53.226920 11409 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 17:02:53.227183 11409 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 17:02:53.228902 11326 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4d9c9f08ec054f01b6d8472bb31c162d [term 1 FOLLOWER]: Refusing update from remote peer 28f3d6f2644f47089feb42fb2517308b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 17:02:53.229225 11405 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 28f3d6f2644f47089feb42fb2517308b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d9c9f08ec054f01b6d8472bb31c162d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 45237 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 17:02:53.230593 11371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:53.212748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58744 (local address 127.20.231.188:45237)
0619 17:02:53.212788 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:53.212791 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:53.222403 (+ 9612us) server_negotiation.cc:407] Connection header received
0619 17:02:53.222499 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:53.222500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:53.222512 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:53.222541 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:53.223169 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:53.223938 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:53.224923 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:53.225252 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:53.225335 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:53.226891 (+ 1556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:53.226893 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:53.226893 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:53.226901 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:53.228763 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:53.229283 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:53.229284 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:53.229284 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:53.229346 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:53.229688 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:53.229689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:53.229689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:53.229840 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:53.229962 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:53.230423 (+ 461us) server_negotiation.cc:299] Negotiation successful
0619 17:02:53.230508 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 17:02:53.231135 11358 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d9c9f08ec054f01b6d8472bb31c162d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28f3d6f2644f47089feb42fb2517308b. Latest consensus state: current_term: 1 leader_uuid: "28f3d6f2644f47089feb42fb2517308b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28f3d6f2644f47089feb42fb2517308b" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37111 } } peers { permanent_uuid: "60e035391ed04b9794d9d51903438029" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38845 } } peers { permanent_uuid: "4d9c9f08ec054f01b6d8472bb31c162d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 45237 } } }
I20260619 17:02:53.231231 11358 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d9c9f08ec054f01b6d8472bb31c162d [sys.catalog]: This master's current role is: FOLLOWER
I20260619 17:02:53.231698 11251 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 60e035391ed04b9794d9d51903438029 [term 1 FOLLOWER]: Refusing update from remote peer 28f3d6f2644f47089feb42fb2517308b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 17:02:53.231993 11405 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 28f3d6f2644f47089feb42fb2517308b [LEADER]: Connected to new peer: Peer: permanent_uuid: "60e035391ed04b9794d9d51903438029" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 17:02:53.232699 11405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28f3d6f2644f47089feb42fb2517308b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "28f3d6f2644f47089feb42fb2517308b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28f3d6f2644f47089feb42fb2517308b" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37111 } } peers { permanent_uuid: "60e035391ed04b9794d9d51903438029" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38845 } } peers { permanent_uuid: "4d9c9f08ec054f01b6d8472bb31c162d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 45237 } } }
I20260619 17:02:53.232785 11405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28f3d6f2644f47089feb42fb2517308b [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:53.233098 11358 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d9c9f08ec054f01b6d8472bb31c162d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "28f3d6f2644f47089feb42fb2517308b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28f3d6f2644f47089feb42fb2517308b" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37111 } } peers { permanent_uuid: "60e035391ed04b9794d9d51903438029" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38845 } } peers { permanent_uuid: "4d9c9f08ec054f01b6d8472bb31c162d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 45237 } } }
I20260619 17:02:53.233172 11358 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d9c9f08ec054f01b6d8472bb31c162d [sys.catalog]: This master's current role is: FOLLOWER
I20260619 17:02:53.233230 11373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60e035391ed04b9794d9d51903438029 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 28f3d6f2644f47089feb42fb2517308b. Latest consensus state: current_term: 1 leader_uuid: "28f3d6f2644f47089feb42fb2517308b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28f3d6f2644f47089feb42fb2517308b" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37111 } } peers { permanent_uuid: "60e035391ed04b9794d9d51903438029" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38845 } } peers { permanent_uuid: "4d9c9f08ec054f01b6d8472bb31c162d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 45237 } } }
I20260619 17:02:53.233319 11373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60e035391ed04b9794d9d51903438029 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 17:02:53.233444 11410 mvcc.cc:205] Tried to move back new op lower bound from 7298615595942596608 to 7298615595882323968. Current Snapshot: MvccSnapshot[applied={T|T < 7298615595942596608}]
I20260619 17:02:53.233959 11411 mvcc.cc:205] Tried to move back new op lower bound from 7298615595942596608 to 7298615595882323968. Current Snapshot: MvccSnapshot[applied={T|T < 7298615595942596608}]
I20260619 17:02:53.235308 11409 catalog_manager.cc:1347] Generated new cluster ID: 5bea9ff65a0c4dd5afff54c3a709ae4f
I20260619 17:02:53.235363 11409 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 17:02:53.252223 11387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28f3d6f2644f47089feb42fb2517308b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "28f3d6f2644f47089feb42fb2517308b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28f3d6f2644f47089feb42fb2517308b" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37111 } } peers { permanent_uuid: "60e035391ed04b9794d9d51903438029" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38845 } } peers { permanent_uuid: "4d9c9f08ec054f01b6d8472bb31c162d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 45237 } } }
I20260619 17:02:53.252343 11387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28f3d6f2644f47089feb42fb2517308b [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:53.252811 11373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60e035391ed04b9794d9d51903438029 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "28f3d6f2644f47089feb42fb2517308b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "28f3d6f2644f47089feb42fb2517308b" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 37111 } } peers { permanent_uuid: "60e035391ed04b9794d9d51903438029" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38845 } } peers { permanent_uuid: "4d9c9f08ec054f01b6d8472bb31c162d" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 45237 } } }
I20260619 17:02:53.252902 11373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60e035391ed04b9794d9d51903438029 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 17:02:53.260468 11409 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 17:02:53.261242 11409 catalog_manager.cc:1504] Loading token signing keys...
I20260619 17:02:53.272785 11409 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 28f3d6f2644f47089feb42fb2517308b: Generated new TSK 0
I20260619 17:02:53.273026 11409 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 17:02:53.307734 11205 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 17:02:53.305706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47040 (local address 127.20.231.190:37111)
0619 17:02:53.305755 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:53.305758 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:53.306764 (+ 1006us) server_negotiation.cc:407] Connection header received
0619 17:02:53.306970 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:53.306971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:53.306986 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:53.307022 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:53.307621 (+ 599us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47040: BlockingRecv error: recv got EOF from 127.0.0.1:47040 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260619 17:02:53.308849 11281 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 17:02:53.306128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49740 (local address 127.20.231.189:38845)
0619 17:02:53.306172 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:53.306174 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:53.308038 (+ 1864us) server_negotiation.cc:407] Connection header received
0619 17:02:53.308281 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:53.308282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:53.308296 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:53.308328 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:53.308778 (+ 450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49740: BlockingRecv error: recv got EOF from 127.0.0.1:49740 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260619 17:02:53.309758 11371 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 17:02:53.306465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58748 (local address 127.20.231.188:45237)
0619 17:02:53.306506 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:53.306509 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:53.308159 (+ 1650us) server_negotiation.cc:407] Connection header received
0619 17:02:53.309049 (+ 890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:53.309050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:53.309064 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:53.309105 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:53.309679 (+ 574us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58748: BlockingRecv error: recv got EOF from 127.0.0.1:58748 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 17:02:53.312999 11281 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 17:02:53.311963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49742 (local address 127.20.231.189:38845)
0619 17:02:53.312021 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:53.312023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:53.312275 (+ 252us) server_negotiation.cc:407] Connection header received
0619 17:02:53.312413 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:53.312415 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:53.312437 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:53.312478 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:53.312939 (+ 461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49742: BlockingRecv error: recv got EOF from 127.0.0.1:49742 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260619 17:02:53.316536 11281 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 17:02:53.315591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49756 (local address 127.20.231.189:38845)
0619 17:02:53.315634 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:53.315636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:53.315913 (+ 277us) server_negotiation.cc:407] Connection header received
0619 17:02:53.316020 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:53.316022 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:53.316034 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:53.316067 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:53.316474 (+ 407us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49756: BlockingRecv error: recv got EOF from 127.0.0.1:49756 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260619 17:02:53.320825 11281 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 17:02:53.319891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49768 (local address 127.20.231.189:38845)
0619 17:02:53.319938 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:53.319940 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:53.320217 (+ 277us) server_negotiation.cc:407] Connection header received
0619 17:02:53.320322 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:53.320323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:53.320335 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:53.320365 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:53.320766 (+ 401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49768: BlockingRecv error: recv got EOF from 127.0.0.1:49768 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260619 17:02:53.327351 11281 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 17:02:53.326317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49774 (local address 127.20.231.189:38845)
0619 17:02:53.326365 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:53.326368 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:53.326679 (+ 311us) server_negotiation.cc:407] Connection header received
0619 17:02:53.326787 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:53.326788 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:53.326801 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:53.326833 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:53.327286 (+ 453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49774: BlockingRecv error: recv got EOF from 127.0.0.1:49774 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260619 17:02:53.337952 11281 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 17:02:53.336869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49776 (local address 127.20.231.189:38845)
0619 17:02:53.336923 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:53.336925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:53.337247 (+ 322us) server_negotiation.cc:407] Connection header received
0619 17:02:53.337370 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:53.337371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:53.337386 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:53.337428 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:53.337878 (+ 450us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49776: BlockingRecv error: recv got EOF from 127.0.0.1:49776 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260619 17:02:53.356653 11281 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 17:02:53.355617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49784 (local address 127.20.231.189:38845)
0619 17:02:53.355664 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:53.355667 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:53.355962 (+ 295us) server_negotiation.cc:407] Connection header received
0619 17:02:53.356072 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:53.356073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:53.356086 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:53.356117 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:53.356589 (+ 472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49784: BlockingRecv error: recv got EOF from 127.0.0.1:49784 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260619 17:02:53.391000 11281 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 17:02:53.390143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49790 (local address 127.20.231.189:38845)
0619 17:02:53.390196 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:53.390198 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:53.390346 (+ 148us) server_negotiation.cc:407] Connection header received
0619 17:02:53.390457 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:53.390458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:53.390471 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:53.390547 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:53.390926 (+ 379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49790: BlockingRecv error: recv got EOF from 127.0.0.1:49790 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260619 17:02:53.457357 11281 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 17:02:53.456516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49792 (local address 127.20.231.189:38845)
0619 17:02:53.456629 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:53.456631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:53.456642 (+ 11us) server_negotiation.cc:407] Connection header received
0619 17:02:53.456717 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:53.456718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:53.456730 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:53.456784 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:53.457288 (+ 504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49792: BlockingRecv error: recv got EOF from 127.0.0.1:49792 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260619 17:02:53.587903 11281 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 17:02:53.586907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49796 (local address 127.20.231.189:38845)
0619 17:02:53.586984 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:53.586987 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:53.587219 (+ 232us) server_negotiation.cc:407] Connection header received
0619 17:02:53.587253 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:53.587254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:53.587266 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:53.587369 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:53.587839 (+ 470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49796: BlockingRecv error: recv got EOF from 127.0.0.1:49796 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260619 17:02:53.846433 11281 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 17:02:53.845416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49804 (local address 127.20.231.189:38845)
0619 17:02:53.845605 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:53.845607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:53.845715 (+ 108us) server_negotiation.cc:407] Connection header received
0619 17:02:53.845806 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:53.845808 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:53.845819 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:53.845871 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:53.846370 (+ 499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49804: BlockingRecv error: recv got EOF from 127.0.0.1:49804 (error 108)
Metrics: {"server-negotiator.queue_time_us":155}
I20260619 17:02:54.161859 11369 catalog_manager.cc:1259] Loaded cluster ID: 5bea9ff65a0c4dd5afff54c3a709ae4f
I20260619 17:02:54.161970 11369 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4d9c9f08ec054f01b6d8472bb31c162d: loading cluster ID for follower catalog manager: success
I20260619 17:02:54.162895 11369 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4d9c9f08ec054f01b6d8472bb31c162d: acquiring CA information for follower catalog manager: success
I20260619 17:02:54.163180 11369 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4d9c9f08ec054f01b6d8472bb31c162d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 17:02:54.207474 11384 catalog_manager.cc:1259] Loaded cluster ID: 5bea9ff65a0c4dd5afff54c3a709ae4f
I20260619 17:02:54.207567 11384 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 60e035391ed04b9794d9d51903438029: loading cluster ID for follower catalog manager: success
I20260619 17:02:54.208480 11384 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 60e035391ed04b9794d9d51903438029: acquiring CA information for follower catalog manager: success
I20260619 17:02:54.208770 11384 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 60e035391ed04b9794d9d51903438029: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 17:02:54.363737 11484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:54.360431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49820 (local address 127.20.231.189:38845)
0619 17:02:54.360554 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:54.360556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:54.360564 (+ 8us) server_negotiation.cc:407] Connection header received
0619 17:02:54.360647 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:54.360648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:54.360729 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:54.360822 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 17:02:54.361016 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:54.361644 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:54.362602 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:54.362868 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:54.362991 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:54.363275 (+ 284us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 17:02:54.363431 (+ 156us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 17:02:54.363553 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:54.363594 (+ 41us) server_negotiation.cc:299] Negotiation successful
0619 17:02:54.363624 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":68,"threads_started":1}
I20260619 17:02:54.369690 11490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:54.366251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58752 (local address 127.20.231.188:45237)
0619 17:02:54.366394 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:54.366396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:54.366407 (+ 11us) server_negotiation.cc:407] Connection header received
0619 17:02:54.366469 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:54.366470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:54.366508 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:54.366579 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 17:02:54.366895 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:54.367644 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:54.368678 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:54.368912 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:54.368987 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:54.369194 (+ 207us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 17:02:54.369320 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 17:02:54.369428 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:54.369525 (+ 97us) server_negotiation.cc:299] Negotiation successful
0619 17:02:54.369548 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":82,"threads_started":1}
I20260619 17:02:54.375938 11484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:54.372285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49834 (local address 127.20.231.189:38845)
0619 17:02:54.372353 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:54.372355 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:54.372449 (+ 94us) server_negotiation.cc:407] Connection header received
0619 17:02:54.372589 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:54.372591 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:54.372607 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:54.372640 (+ 33us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 17:02:54.372998 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:54.373917 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:54.374895 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:54.375252 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:54.375336 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:54.375619 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 17:02:54.375659 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 17:02:54.375716 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:54.375797 (+ 81us) server_negotiation.cc:299] Negotiation successful
0619 17:02:54.375827 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 17:02:54.376006 11490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:54.372568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58754 (local address 127.20.231.188:45237)
0619 17:02:54.372617 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:54.372620 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:54.372758 (+ 138us) server_negotiation.cc:407] Connection header received
0619 17:02:54.372868 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:54.372870 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:54.372884 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:54.372920 (+ 36us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 17:02:54.373142 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:54.373668 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:54.374959 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:54.375259 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:54.375367 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:54.375628 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 17:02:54.375666 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 17:02:54.375729 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:54.375853 (+ 124us) server_negotiation.cc:299] Negotiation successful
0619 17:02:54.375881 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 17:02:54.376494 11496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:54.372155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47048 (local address 127.20.231.190:37111)
0619 17:02:54.372294 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:54.372296 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:54.372308 (+ 12us) server_negotiation.cc:407] Connection header received
0619 17:02:54.372527 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:54.372529 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:54.372594 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:54.372724 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 17:02:54.372897 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:54.373834 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:54.374948 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:54.375294 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:54.375360 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:54.375623 (+ 263us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 17:02:54.375755 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 17:02:54.375821 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:54.376001 (+ 180us) server_negotiation.cc:299] Negotiation successful
0619 17:02:54.376393 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":71,"threads_started":1}
I20260619 17:02:54.379148 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 11134
I20260619 17:02:54.384815 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 11209
I20260619 17:02:54.390096 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 11284
2026-06-19T17:02:54Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2982 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 17:02:54 dist-test-slave-vnbj krb5kdc[11501](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 17:02:54 dist-test-slave-vnbj krb5kdc[11501](info): set up 2 sockets
Jun 19 17:02:54 dist-test-slave-vnbj krb5kdc[11501](info): commencing operation
krb5kdc: starting...
W20260619 17:02:55.447882 21406 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/krb5kdc/test-user.keytab.
Jun 19 17:02:55 dist-test-slave-vnbj krb5kdc[11501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T17:02:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T17:02:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.20.231.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
Entry for principal kudu/127.20.231.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab.
I20260619 17:02:55.597095 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:38663
--webserver_interface=127.20.231.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:35273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:38663,127.20.231.189:38811,127.20.231.188:42773
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 17:02:55.689831 11514 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 17:02:55.690100 11514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:55.690119 11514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:55.690142 11514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:55.691550 11514 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:02:55.691610 11514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:55.691624 11514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:55.691638 11514 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:02:55.691651 11514 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:02:55.693339 11514 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:35273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:38663,127.20.231.189:38811,127.20.231.188:42773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.190.keytab
--principal=kudu/127.20.231.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.190:38663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.20.231.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11514
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:55.693651 11514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:55.693934 11514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:55.699846 11522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:55.699843 11520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:55.699843 11519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:55.699865 11514 server_base.cc:1034] running on GCE node
I20260619 17:02:55.700541 11514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:55.701041 11514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:55.702251 11514 hybrid_clock.cc:648] HybridClock initialized: now 1781888575702245 us; error 48 us; skew 500 ppm
Jun 19 17:02:55 dist-test-slave-vnbj krb5kdc[11501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888575, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:55.710084 11514 init.cc:377] Logged in from keytab as kudu/127.20.231.190@KRBTEST.COM (short username kudu)
I20260619 17:02:55.710592 11514 webserver.cc:460] Webserver started at http://127.20.231.190:44525/ using document root <none> and password file <none>
I20260619 17:02:55.710808 11514 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:55.710927 11514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:55.711071 11514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:55.712332 11514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-0/data/instance:
uuid: "b6e2f9d10fb448f5826a5906f54d8f51"
format_stamp: "Formatted at 2026-06-19 17:02:55 on dist-test-slave-vnbj"
server_key: "3d3636356a47cc887741a53a82484ef8"
server_key_iv: "403bf6837e63fc9aa673f25eb7ff694c"
server_key_version: "encryptionkey@0"
I20260619 17:02:55.712709 11514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-0/wal/instance:
uuid: "b6e2f9d10fb448f5826a5906f54d8f51"
format_stamp: "Formatted at 2026-06-19 17:02:55 on dist-test-slave-vnbj"
server_key: "3d3636356a47cc887741a53a82484ef8"
server_key_iv: "403bf6837e63fc9aa673f25eb7ff694c"
server_key_version: "encryptionkey@0"
I20260619 17:02:55.714527 11514 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260619 17:02:55.715704 11529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:55.715911 11514 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 17:02:55.716025 11514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-0/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-0/wal
uuid: "b6e2f9d10fb448f5826a5906f54d8f51"
format_stamp: "Formatted at 2026-06-19 17:02:55 on dist-test-slave-vnbj"
server_key: "3d3636356a47cc887741a53a82484ef8"
server_key_iv: "403bf6837e63fc9aa673f25eb7ff694c"
server_key_version: "encryptionkey@0"
I20260619 17:02:55.716104 11514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:55.725344 11514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:55.728169 11514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:55.728318 11514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:55.732788 11514 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.190:38663
I20260619 17:02:55.732905 11581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.190:38663 every 8 connection(s)
I20260619 17:02:55.733330 11514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-0/data/info.pb
I20260619 17:02:55.734375 11582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:55.736138 11582 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.231.190" port: 38663 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 17:02:55 dist-test-slave-vnbj krb5kdc[11501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888575, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:02:55.743549 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 11514
I20260619 17:02:55.743659 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-0/wal/instance
I20260619 17:02:55.743911 21406 external_mini_cluster.cc:1428] Setting key 171c1c1f406de6a25d6b8f10a86264d2
I20260619 17:02:55.749429 11585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:55.737051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56176 (local address 127.20.231.190:38663)
0619 17:02:55.737261 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:55.737266 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:55.737299 (+ 33us) server_negotiation.cc:407] Connection header received
0619 17:02:55.737927 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:55.737929 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:55.737951 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:55.738044 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:55.738665 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:55.739690 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:55.740624 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:55.741073 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:55.741185 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:55.743917 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:55.743927 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:55.743969 (+ 42us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:55.743979 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:55.747104 (+ 3125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:55.747738 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:55.747742 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:55.747743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:55.747881 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:55.748242 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:55.748243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:55.748244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:55.748545 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:55.748633 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:55.749035 (+ 402us) server_negotiation.cc:299] Negotiation successful
0619 17:02:55.749219 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":178,"thread_start_us":100,"threads_started":1}
I20260619 17:02:55.749607 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:55.736888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:38663 (local address 127.0.0.1:56176)
0619 17:02:55.737123 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:55.737181 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:55.737729 (+ 548us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:55.738080 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:55.738083 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:55.738346 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:55.738543 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:55.738548 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:55.739791 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:55.739792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:55.740521 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:55.740524 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:55.741185 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:55.741186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:55.741259 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:55.741913 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:02:55.741923 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:55.743778 (+ 1855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:55.747219 (+ 3441us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:55.747231 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:55.747234 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:55.747598 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:55.747978 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:55.747979 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:55.747979 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:55.748114 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:55.748641 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:55.748650 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:55.748748 (+ 98us) client_negotiation.cc:769] Sending connection context
0619 17:02:55.748979 (+ 231us) client_negotiation.cc:240] Negotiation successful
0619 17:02:55.749341 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":109,"thread_start_us":97,"threads_started":1}
I20260619 17:02:55.750553 11582 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38811 } has no permanent_uuid. Determining permanent_uuid...
I20260619 17:02:55.751040 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:55.750839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.189:38811 (local address 127.0.0.1:39826)
0619 17:02:55.750931 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:55.751000 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.189:38811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260619 17:02:55.751252 11533 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.231.189:38811: connect: Connection refused (error 111)
I20260619 17:02:55.751852 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:55.751708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.189:38811 (local address 127.0.0.1:39834)
0619 17:02:55.751782 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:55.751817 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.189:38811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260619 17:02:55.752051 11582 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.231.189:38811: Network error: Client connection negotiation failed: client connection to 127.20.231.189:38811: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.20.231.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.189.keytab.
Entry for principal kudu/127.20.231.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.189.keytab.
I20260619 17:02:55.782656 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.189:38811
--webserver_interface=127.20.231.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.189.keytab
--principal=kudu/127.20.231.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:35273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:38663,127.20.231.189:38811,127.20.231.188:42773
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 17:02:55.801333 11582 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38811 } attempt: 1
I20260619 17:02:55.801923 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:55.801766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.189:38811 (local address 127.0.0.1:39840)
0619 17:02:55.801834 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:55.801883 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.189:38811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 17:02:55.802338 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:55.802235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.189:38811 (local address 127.0.0.1:39852)
0619 17:02:55.802286 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:55.802316 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.189:38811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 17:02:55.802500 11582 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.231.189:38811: Network error: Client connection negotiation failed: client connection to 127.20.231.189:38811: connect: Connection refused (error 111)
I20260619 17:02:55.870733 11582 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38811 } attempt: 2
I20260619 17:02:55.871537 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:55.871240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.189:38811 (local address 127.0.0.1:39864)
0619 17:02:55.871455 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:55.871503 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.189:38811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181}
I20260619 17:02:55.872064 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:55.871909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.189:38811 (local address 127.0.0.1:39872)
0619 17:02:55.872003 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:55.872038 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.189:38811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260619 17:02:55.872234 11582 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.231.189:38811: Network error: Client connection negotiation failed: client connection to 127.20.231.189:38811: connect: Connection refused (error 111)
W20260619 17:02:55.877889 11589 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 17:02:55.878185 11589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:55.878226 11589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:55.878291 11589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:55.879824 11589 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:02:55.879914 11589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:55.879947 11589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:55.879980 11589 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:02:55.880005 11589 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:02:55.881801 11589 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:35273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:38663,127.20.231.189:38811,127.20.231.188:42773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.189.keytab
--principal=kudu/127.20.231.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.189:38811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.20.231.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:55.882135 11589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:55.882382 11589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:55.887934 11595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:55.887902 11597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:55.887904 11594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:55.888442 11589 server_base.cc:1034] running on GCE node
I20260619 17:02:55.888737 11589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:55.889146 11589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:55.890386 11589 hybrid_clock.cc:648] HybridClock initialized: now 1781888575890340 us; error 86 us; skew 500 ppm
Jun 19 17:02:55 dist-test-slave-vnbj krb5kdc[11501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888575, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:55.893324 11589 init.cc:377] Logged in from keytab as kudu/127.20.231.189@KRBTEST.COM (short username kudu)
I20260619 17:02:55.893882 11589 webserver.cc:460] Webserver started at http://127.20.231.189:44125/ using document root <none> and password file <none>
I20260619 17:02:55.894083 11589 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:55.894126 11589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:55.894220 11589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:55.895483 11589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-1/data/instance:
uuid: "035b5a44b0434ecb9eb79d53fd9d92c0"
format_stamp: "Formatted at 2026-06-19 17:02:55 on dist-test-slave-vnbj"
server_key: "997ff7f12a8e2a190a20274b2cfb6a62"
server_key_iv: "b097b0dd5ca27f085cb9884b3a8fe97b"
server_key_version: "encryptionkey@0"
I20260619 17:02:55.895825 11589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-1/wal/instance:
uuid: "035b5a44b0434ecb9eb79d53fd9d92c0"
format_stamp: "Formatted at 2026-06-19 17:02:55 on dist-test-slave-vnbj"
server_key: "997ff7f12a8e2a190a20274b2cfb6a62"
server_key_iv: "b097b0dd5ca27f085cb9884b3a8fe97b"
server_key_version: "encryptionkey@0"
I20260619 17:02:55.897362 11589 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260619 17:02:55.898231 11604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:55.898442 11589 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 17:02:55.898551 11589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-1/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-1/wal
uuid: "035b5a44b0434ecb9eb79d53fd9d92c0"
format_stamp: "Formatted at 2026-06-19 17:02:55 on dist-test-slave-vnbj"
server_key: "997ff7f12a8e2a190a20274b2cfb6a62"
server_key_iv: "b097b0dd5ca27f085cb9884b3a8fe97b"
server_key_version: "encryptionkey@0"
I20260619 17:02:55.898628 11589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:55.906500 11589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:55.909051 11589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:55.909179 11589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:55.913339 11589 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.189:38811
I20260619 17:02:55.913414 11656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.189:38811 every 8 connection(s)
I20260619 17:02:55.913812 11589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-1/data/info.pb
I20260619 17:02:55.915000 11657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:55.916728 11657 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.231.190" port: 38663 } has no permanent_uuid. Determining permanent_uuid...
I20260619 17:02:55.919638 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 11589
I20260619 17:02:55.919756 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-1/wal/instance
I20260619 17:02:55.919991 21406 external_mini_cluster.cc:1428] Setting key b355dddb00a40033200a0d6106d14048
Jun 19 17:02:55 dist-test-slave-vnbj krb5kdc[11501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888575, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.189@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.20.231.188@KRBTEST.COM; defaulting to no policy
I20260619 17:02:55.929157 11585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:55.917773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56186 (local address 127.20.231.190:38663)
0619 17:02:55.917902 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:55.917904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:55.918099 (+ 195us) server_negotiation.cc:407] Connection header received
0619 17:02:55.918659 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:55.918660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:55.918673 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:55.918738 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:55.919740 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:55.920597 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:55.922074 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:55.922357 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:55.922425 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:55.925452 (+ 3027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:55.925454 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:55.925454 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:55.925464 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:55.927110 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:55.927797 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:55.927798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:55.927799 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:55.927888 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:55.928275 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:55.928276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:55.928277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:55.928454 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:55.928591 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:55.928991 (+ 400us) server_negotiation.cc:299] Negotiation successful
0619 17:02:55.929078 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260619 17:02:55.929306 11659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:55.917633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:38663 (local address 127.0.0.1:56186)
0619 17:02:55.917940 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:55.917983 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:55.918530 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:55.918991 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:55.918996 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:55.919309 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:55.919625 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:55.919631 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:55.920721 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:55.920722 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:55.921951 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:55.921956 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:55.922461 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:55.922463 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:55.922636 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:55.923340 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:02:55.923349 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:55.925287 (+ 1938us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:55.927218 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:55.927230 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:55.927233 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:55.927635 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:55.927980 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:55.927982 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:55.927982 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:55.928162 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:55.928594 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:55.928604 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:55.928818 (+ 214us) client_negotiation.cc:769] Sending connection context
0619 17:02:55.928933 (+ 115us) client_negotiation.cc:240] Negotiation successful
0619 17:02:55.929083 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":102,"threads_started":1}
I20260619 17:02:55.930025 11657 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38811 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.20.231.188@KRBTEST.COM" created.
Jun 19 17:02:55 dist-test-slave-vnbj krb5kdc[11501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888575, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.189@KRBTEST.COM for kudu/127.20.231.189@KRBTEST.COM
I20260619 17:02:55.939558 11659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:55.930334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.189:38811 (local address 127.0.0.1:39882)
0619 17:02:55.930416 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:55.930428 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:55.930479 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:55.930830 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:55.930831 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:55.931127 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:55.931183 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:55.931186 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:55.932301 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:55.932303 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:55.932824 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:55.932826 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:55.933420 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:55.933421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:55.933488 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:55.933971 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:02:55.933977 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:55.935516 (+ 1539us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:55.938083 (+ 2567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:55.938085 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:55.938085 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:55.938359 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:55.938670 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:55.938671 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:55.938671 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:55.938745 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:55.939239 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:55.939240 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:55.939292 (+ 52us) client_negotiation.cc:769] Sending connection context
0619 17:02:55.939380 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 17:02:55.939481 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260619 17:02:55.939697 11661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:55.930496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39882 (local address 127.20.231.189:38811)
0619 17:02:55.930677 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:55.930681 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:55.930698 (+ 17us) server_negotiation.cc:407] Connection header received
0619 17:02:55.930717 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:55.930719 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:55.930730 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:55.930828 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:55.931338 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:55.932209 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:55.932975 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:55.933319 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:55.933422 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:55.935618 (+ 2196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:55.935630 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:55.935630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:55.935639 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:55.937948 (+ 2309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:55.938483 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:55.938487 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:55.938488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:55.938564 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:55.938837 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:55.938839 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:55.938839 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:55.939120 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:55.939249 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:55.939464 (+ 215us) server_negotiation.cc:299] Negotiation successful
0619 17:02:55.939569 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":98,"threads_started":1}
I20260619 17:02:55.940495 11657 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.231.188" port: 42773 } has no permanent_uuid. Determining permanent_uuid...
I20260619 17:02:55.941036 11659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:55.940895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:42773 (local address 127.0.0.1:33688)
0619 17:02:55.940953 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:55.941009 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.188:42773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260619 17:02:55.941226 11605 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.20.231.188:42773: connect: Connection refused (error 111)
I20260619 17:02:55.941893 11659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:55.941668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:42773 (local address 127.0.0.1:33696)
0619 17:02:55.941746 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:55.941867 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.188:42773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260619 17:02:55.942126 11657 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.231.188:42773: Network error: Client connection negotiation failed: client connection to 127.20.231.188:42773: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.20.231.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.188.keytab.
Entry for principal kudu/127.20.231.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.188.keytab.
I20260619 17:02:55.956188 21406 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGfxJfS/build/release/bin/kudu
/tmp/dist-test-taskGfxJfS/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.188:42773
--webserver_interface=127.20.231.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.188.keytab
--principal=kudu/127.20.231.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.20.231.148:35273
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.20.231.190:38663,127.20.231.189:38811,127.20.231.188:42773
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 17:02:55.963450 11582 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38811 } attempt: 3
Jun 19 17:02:55 dist-test-slave-vnbj krb5kdc[11501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888575, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for kudu/127.20.231.189@KRBTEST.COM
I20260619 17:02:55.972949 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:55.964029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.189:38811 (local address 127.0.0.1:39898)
0619 17:02:55.964106 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:55.964118 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:55.964218 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:55.964436 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:55.964438 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:55.964661 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:55.964732 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:55.964735 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:55.965460 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:55.965462 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:55.966038 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:55.966041 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:55.966534 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:55.966535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:55.966630 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:55.967205 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:02:55.967210 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:55.968817 (+ 1607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:55.971506 (+ 2689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:55.971507 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:55.971508 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:55.971767 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:55.972045 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:55.972046 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:55.972046 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:55.972135 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:55.972544 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:55.972545 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:55.972613 (+ 68us) client_negotiation.cc:769] Sending connection context
0619 17:02:55.972701 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 17:02:55.972831 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260619 17:02:55.972980 11661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:55.964116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39898 (local address 127.20.231.189:38811)
0619 17:02:55.964232 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:55.964234 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:55.964248 (+ 14us) server_negotiation.cc:407] Connection header received
0619 17:02:55.964281 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:55.964282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:55.964294 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:55.964373 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:55.964830 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:55.965359 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:55.966191 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:55.966436 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:55.966533 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:55.969049 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:55.969050 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:55.969051 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:55.969067 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:55.971312 (+ 2245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:55.971871 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:55.971873 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:55.971874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:55.971959 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:55.972251 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:55.972253 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:55.972253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:55.972444 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:55.972532 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:55.972743 (+ 211us) server_negotiation.cc:299] Negotiation successful
0619 17:02:55.972846 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260619 17:02:55.973932 11582 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.231.188" port: 42773 } has no permanent_uuid. Determining permanent_uuid...
I20260619 17:02:55.974327 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:55.974209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:42773 (local address 127.0.0.1:33712)
0619 17:02:55.974261 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:55.974293 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.188:42773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 17:02:55.974712 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:55.974616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:42773 (local address 127.0.0.1:33728)
0619 17:02:55.974664 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:55.974690 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.188:42773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260619 17:02:55.974810 11582 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.231.188:42773: Network error: Client connection negotiation failed: client connection to 127.20.231.188:42773: connect: Connection refused (error 111)
I20260619 17:02:55.991360 11657 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.231.188" port: 42773 } attempt: 1
I20260619 17:02:55.991890 11659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:55.991722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:42773 (local address 127.0.0.1:33736)
0619 17:02:55.991795 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:55.991847 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.188:42773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260619 17:02:55.992331 11659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:55.992228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:42773 (local address 127.0.0.1:33748)
0619 17:02:55.992278 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:55.992309 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.188:42773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260619 17:02:55.992493 11657 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.231.188:42773: Network error: Client connection negotiation failed: client connection to 127.20.231.188:42773: connect: Connection refused (error 111)
I20260619 17:02:56.006004 11582 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.231.188" port: 42773 } attempt: 1
I20260619 17:02:56.006537 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:56.006365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:42773 (local address 127.0.0.1:33762)
0619 17:02:56.006441 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:56.006496 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.188:42773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260619 17:02:56.006950 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:56.006830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:42773 (local address 127.0.0.1:33770)
0619 17:02:56.006891 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:56.006928 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.188:42773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260619 17:02:56.007109 11582 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.231.188:42773: Network error: Client connection negotiation failed: client connection to 127.20.231.188:42773: connect: Connection refused (error 111)
W20260619 17:02:56.047192 11664 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 17:02:56.047482 11664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 17:02:56.047504 11664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 17:02:56.047528 11664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 17:02:56.048909 11664 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 17:02:56.048974 11664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 17:02:56.048988 11664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 17:02:56.049002 11664 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 17:02:56.049014 11664 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 17:02:56.050720 11664 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.20.231.148:35273
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.20.231.190:38663,127.20.231.189:38811,127.20.231.188:42773
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/krb5kdc/kudu_127.20.231.188.keytab
--principal=kudu/127.20.231.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.20.231.188:42773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.20.231.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 19 Jun 2026 15:28:00 UTC on ccycloud.quasar-opyasf.root.comops.site
build id 4455770
I20260619 17:02:56.051091 11664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 17:02:56.051364 11664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 17:02:56.056917 11670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:56.057052 11672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 17:02:56.056923 11669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 17:02:56.057384 11664 server_base.cc:1034] running on GCE node
I20260619 17:02:56.057796 11664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 17:02:56.058172 11664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 17:02:56.059396 11664 hybrid_clock.cc:648] HybridClock initialized: now 1781888576059376 us; error 33 us; skew 500 ppm
I20260619 17:02:56.060729 11657 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.231.188" port: 42773 } attempt: 2
I20260619 17:02:56.061440 11659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:56.061235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:42773 (local address 127.0.0.1:33772)
0619 17:02:56.061347 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:56.061397 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.188:42773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260619 17:02:56.061918 11659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:56.061792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:42773 (local address 127.0.0.1:33774)
0619 17:02:56.061862 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:56.061893 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.188:42773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260619 17:02:56.062068 11657 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.231.188:42773: Network error: Client connection negotiation failed: client connection to 127.20.231.188:42773: connect: Connection refused (error 111)
Jun 19 17:02:56 dist-test-slave-vnbj krb5kdc[11501](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888576, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 17:02:56.063181 11664 init.cc:377] Logged in from keytab as kudu/127.20.231.188@KRBTEST.COM (short username kudu)
I20260619 17:02:56.063831 11664 webserver.cc:460] Webserver started at http://127.20.231.188:41501/ using document root <none> and password file <none>
I20260619 17:02:56.064046 11664 fs_manager.cc:359] Metadata directory not provided
I20260619 17:02:56.064095 11664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 17:02:56.064201 11664 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 17:02:56.065433 11664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-2/data/instance:
uuid: "e078cb3b5e104607a36d018f416ecc69"
format_stamp: "Formatted at 2026-06-19 17:02:56 on dist-test-slave-vnbj"
server_key: "da471eb9cebfe8f1149324e108056c64"
server_key_iv: "7f6d7a4238b6e70e7a15cc8aed51161d"
server_key_version: "encryptionkey@0"
I20260619 17:02:56.065850 11664 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-2/wal/instance:
uuid: "e078cb3b5e104607a36d018f416ecc69"
format_stamp: "Formatted at 2026-06-19 17:02:56 on dist-test-slave-vnbj"
server_key: "da471eb9cebfe8f1149324e108056c64"
server_key_iv: "7f6d7a4238b6e70e7a15cc8aed51161d"
server_key_version: "encryptionkey@0"
I20260619 17:02:56.067564 11664 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260619 17:02:56.068586 11679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 17:02:56.068811 11664 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260619 17:02:56.068905 11664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-2/data,/tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-2/wal
uuid: "e078cb3b5e104607a36d018f416ecc69"
format_stamp: "Formatted at 2026-06-19 17:02:56 on dist-test-slave-vnbj"
server_key: "da471eb9cebfe8f1149324e108056c64"
server_key_iv: "7f6d7a4238b6e70e7a15cc8aed51161d"
server_key_version: "encryptionkey@0"
I20260619 17:02:56.068969 11664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 17:02:56.082258 11582 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.231.188" port: 42773 } attempt: 2
I20260619 17:02:56.082795 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:56.082634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:42773 (local address 127.0.0.1:33778)
0619 17:02:56.082680 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:56.082738 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.188:42773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260619 17:02:56.083304 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:56.083185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:42773 (local address 127.0.0.1:33790)
0619 17:02:56.083245 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:56.083278 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.20.231.188:42773: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260619 17:02:56.083457 11582 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.20.231.188:42773: Network error: Client connection negotiation failed: client connection to 127.20.231.188:42773: connect: Connection refused (error 111)
I20260619 17:02:56.084203 11664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 17:02:56.087090 11664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 17:02:56.087252 11664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 17:02:56.091771 11664 rpc_server.cc:307] RPC server started. Bound to: 127.20.231.188:42773
I20260619 17:02:56.092152 11731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.20.231.188:42773 every 8 connection(s)
I20260619 17:02:56.092274 11664 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-2/data/info.pb
I20260619 17:02:56.092972 11732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 17:02:56.094808 11732 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.231.190" port: 38663 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 17:02:56 dist-test-slave-vnbj krb5kdc[11501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888576, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.188@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
I20260619 17:02:56.102375 21406 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGfxJfS/build/release/bin/kudu as pid 11664
I20260619 17:02:56.102483 21406 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGfxJfS/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781888301345144-21406-0/minicluster-data/master-2/wal/instance
I20260619 17:02:56.102733 21406 external_mini_cluster.cc:1428] Setting key f06d3493e495c2db3eb90ecb222f464e
I20260619 17:02:56.106550 11585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:56.095827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56192 (local address 127.20.231.190:38663)
0619 17:02:56.095912 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:56.095915 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:56.096129 (+ 214us) server_negotiation.cc:407] Connection header received
0619 17:02:56.096717 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:56.096718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:56.096730 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:56.096792 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:56.097511 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:56.098111 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:56.099365 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:56.099583 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:56.099672 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:56.102680 (+ 3008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:56.102682 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:56.102682 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:56.102691 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:56.104764 (+ 2073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:56.105326 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:56.105327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:56.105327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:56.105396 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:56.105737 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:56.105738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:56.105738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:56.105893 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:56.106050 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:56.106355 (+ 305us) server_negotiation.cc:299] Negotiation successful
0619 17:02:56.106443 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 17:02:56.106817 11734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:56.095695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.190:38663 (local address 127.0.0.1:56192)
0619 17:02:56.095987 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:56.096025 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:56.096577 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:56.096884 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:56.096888 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:56.097156 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:56.097403 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:56.097408 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:56.098241 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:56.098242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:56.099255 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:56.099267 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:56.099784 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:56.099786 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:56.099919 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:56.100569 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:02:56.100578 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:56.102557 (+ 1979us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:56.104945 (+ 2388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:56.104954 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:56.104956 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:56.105242 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:56.105471 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:56.105472 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:56.105473 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:56.105642 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:56.106043 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:56.106051 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:56.106203 (+ 152us) client_negotiation.cc:769] Sending connection context
0619 17:02:56.106321 (+ 118us) client_negotiation.cc:240] Negotiation successful
0619 17:02:56.106552 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":157,"threads_started":1}
I20260619 17:02:56.107460 11732 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38811 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 17:02:56 dist-test-slave-vnbj krb5kdc[11501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.188@KRBTEST.COM
Jun 19 17:02:56 dist-test-slave-vnbj krb5kdc[11501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888576, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.188@KRBTEST.COM for kudu/127.20.231.189@KRBTEST.COM
I20260619 17:02:56.113922 11736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:56.103170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33796 (local address 127.20.231.188:42773)
0619 17:02:56.103371 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:56.103376 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:56.103562 (+ 186us) server_negotiation.cc:407] Connection header received
0619 17:02:56.103595 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:56.103599 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:56.103617 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:56.103708 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:56.104566 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:56.105529 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:56.106479 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:56.107080 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:56.107267 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:56.110388 (+ 3121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:56.110402 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:56.110402 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:56.110411 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:56.112584 (+ 2173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:56.113018 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:56.113021 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:56.113022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:56.113085 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:56.113304 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:56.113305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:56.113306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:56.113473 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:56.113660 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:56.113712 (+ 52us) server_negotiation.cc:299] Negotiation successful
0619 17:02:56.113796 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":85,"threads_started":1}
I20260619 17:02:56.116833 11661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:56.108171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39910 (local address 127.20.231.189:38811)
0619 17:02:56.108239 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:56.108241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:56.108299 (+ 58us) server_negotiation.cc:407] Connection header received
0619 17:02:56.108396 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:56.108397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:56.108406 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:56.108450 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:56.108888 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:56.109466 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:56.110464 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:56.110763 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:56.110833 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:56.113175 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:56.113176 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:56.113176 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:56.113184 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:56.114970 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:56.115425 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:56.115426 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:56.115427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:56.115488 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:56.116287 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:56.116288 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:56.116288 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:56.116410 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:56.116494 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:56.116659 (+ 165us) server_negotiation.cc:299] Negotiation successful
0619 17:02:56.116737 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 17:02:56.116833 11734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:56.108084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.189:38811 (local address 127.0.0.1:39910)
0619 17:02:56.108233 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:56.108246 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:56.108308 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:56.108454 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:56.108454 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:56.108631 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:56.108705 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:56.108707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:56.109580 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:56.109581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:56.110259 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:56.110261 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:56.111164 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:56.111165 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:56.111253 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:56.111584 (+ 331us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:02:56.111588 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:56.113070 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:56.115078 (+ 2008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:56.115079 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:56.115080 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:56.115347 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:56.115561 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:56.115561 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:56.115562 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:56.115621 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:56.116486 (+ 865us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:56.116488 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:56.116535 (+ 47us) client_negotiation.cc:769] Sending connection context
0619 17:02:56.116617 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 17:02:56.116725 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125}
I20260619 17:02:56.117250 11732 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.20.231.188" port: 42773 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 17:02:56 dist-test-slave-vnbj krb5kdc[11501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.190@KRBTEST.COM
Jun 19 17:02:56 dist-test-slave-vnbj krb5kdc[11501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888576, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.188@KRBTEST.COM for kudu/127.20.231.188@KRBTEST.COM
I20260619 17:02:56.125378 11585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:56.114848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56208 (local address 127.20.231.190:38663)
0619 17:02:56.115100 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:56.115103 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:56.115112 (+ 9us) server_negotiation.cc:407] Connection header received
0619 17:02:56.115140 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:56.115141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:56.115155 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:56.115213 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:56.115697 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:56.116200 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:56.117434 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:56.117754 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:56.117821 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:56.120808 (+ 2987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:56.120810 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:56.120810 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:56.120820 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:56.123291 (+ 2471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:56.123853 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:56.123855 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:56.123855 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:56.123927 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:56.124159 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:56.124161 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:56.124162 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:56.124353 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:56.124942 (+ 589us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:56.125154 (+ 212us) server_negotiation.cc:299] Negotiation successful
0619 17:02:56.125266 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":205}
I20260619 17:02:56.125451 11736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:56.117557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33808 (local address 127.20.231.188:42773)
0619 17:02:56.117615 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:56.117617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:56.117624 (+ 7us) server_negotiation.cc:407] Connection header received
0619 17:02:56.117685 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:56.117686 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:56.117696 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:56.117741 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:56.118025 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:56.118826 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:56.119731 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:56.119971 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:56.120089 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:56.122206 (+ 2117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:56.122207 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:56.122208 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:56.122215 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:56.124272 (+ 2057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:56.124610 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:56.124611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:56.124612 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:56.124663 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:56.124863 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:56.124864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:56.124864 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:56.124982 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:56.125059 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:56.125187 (+ 128us) server_negotiation.cc:299] Negotiation successful
0619 17:02:56.125329 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":32}
I20260619 17:02:56.125713 11734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:56.117528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:42773 (local address 127.0.0.1:33808)
0619 17:02:56.117590 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:56.117600 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:56.117631 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:56.117745 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:56.117746 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:56.117913 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:56.117952 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:56.117953 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:56.118989 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:56.118991 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:56.119650 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:56.119653 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:56.120091 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:56.120092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:56.120152 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:56.120662 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:02:56.120667 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:56.122087 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:56.124356 (+ 2269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:56.124357 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:56.124358 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:56.124521 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:56.124735 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:56.124736 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:56.124737 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:56.124791 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:56.125050 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:56.125051 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:56.125090 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 17:02:56.125161 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 17:02:56.125657 (+ 496us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":220}
I20260619 17:02:56.127260 11732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e078cb3b5e104607a36d018f416ecc69: Bootstrap starting.
I20260619 17:02:56.128221 11732 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e078cb3b5e104607a36d018f416ecc69: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:56.128536 11732 log.cc:824] T 00000000000000000000000000000000 P e078cb3b5e104607a36d018f416ecc69: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:56.129436 11732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e078cb3b5e104607a36d018f416ecc69: No bootstrap required, opened a new log
Jun 19 17:02:56 dist-test-slave-vnbj krb5kdc[11501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888575, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.20.231.189@KRBTEST.COM
I20260619 17:02:56.131153 11732 raft_consensus.cc:348] T 00000000000000000000000000000000 P e078cb3b5e104607a36d018f416ecc69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e2f9d10fb448f5826a5906f54d8f51" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 38663 } } peers { permanent_uuid: "035b5a44b0434ecb9eb79d53fd9d92c0" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38811 } } peers { permanent_uuid: "e078cb3b5e104607a36d018f416ecc69" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 42773 } }
I20260619 17:02:56.131359 11732 raft_consensus.cc:729] T 00000000000000000000000000000000 P e078cb3b5e104607a36d018f416ecc69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e078cb3b5e104607a36d018f416ecc69, State: Initialized, Role: FOLLOWER
I20260619 17:02:56.131461 11732 consensus_queue.cc:260] T 00000000000000000000000000000000 P e078cb3b5e104607a36d018f416ecc69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e2f9d10fb448f5826a5906f54d8f51" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 38663 } } peers { permanent_uuid: "035b5a44b0434ecb9eb79d53fd9d92c0" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38811 } } peers { permanent_uuid: "e078cb3b5e104607a36d018f416ecc69" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 42773 } }
I20260619 17:02:56.131845 11738 sys_catalog.cc:455] T 00000000000000000000000000000000 P e078cb3b5e104607a36d018f416ecc69 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e2f9d10fb448f5826a5906f54d8f51" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 38663 } } peers { permanent_uuid: "035b5a44b0434ecb9eb79d53fd9d92c0" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38811 } } peers { permanent_uuid: "e078cb3b5e104607a36d018f416ecc69" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 42773 } } }
I20260619 17:02:56.131891 11732 sys_catalog.cc:564] T 00000000000000000000000000000000 P e078cb3b5e104607a36d018f416ecc69 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:02:56.131951 11738 sys_catalog.cc:458] T 00000000000000000000000000000000 P e078cb3b5e104607a36d018f416ecc69 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 17:02:56.134068 11749 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e078cb3b5e104607a36d018f416ecc69: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 17:02:56.134137 11749 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 17:02:56.134742 11661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:56.126173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39914 (local address 127.20.231.189:38811)
0619 17:02:56.126217 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:56.126219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:56.126228 (+ 9us) server_negotiation.cc:407] Connection header received
0619 17:02:56.126255 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:56.126256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:56.126265 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:56.126312 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:56.126828 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:56.127387 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:56.128070 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:56.128331 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:56.128395 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:56.130965 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:56.130967 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:56.130967 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:56.130976 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:56.132660 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:56.133090 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:56.133092 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:56.133092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:56.133159 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:56.133445 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:56.133446 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:56.133447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:56.133616 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:56.134284 (+ 668us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:56.134543 (+ 259us) server_negotiation.cc:299] Negotiation successful
0619 17:02:56.134644 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 17:02:56.136670 21406 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 17:02:56.134454 11664 master_runner.cc:186] Error getting master registration for 127.20.231.190:38663: OK, instance_id { permanent_uuid: "b6e2f9d10fb448f5826a5906f54d8f51" instance_seqno: 1781888575731416 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 17:02:56.137847 11664 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 17:02:56.137764 11556 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b6e2f9d10fb448f5826a5906f54d8f51"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56208
I20260619 17:02:56.153237 11657 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.231.188" port: 42773 } attempt: 3
Jun 19 17:02:56 dist-test-slave-vnbj krb5kdc[11501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888575, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.189@KRBTEST.COM for kudu/127.20.231.188@KRBTEST.COM
I20260619 17:02:56.164407 11736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:56.153816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33812 (local address 127.20.231.188:42773)
0619 17:02:56.153896 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:56.153899 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:56.153910 (+ 11us) server_negotiation.cc:407] Connection header received
0619 17:02:56.153940 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:56.153941 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:56.153961 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:56.154025 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:56.154402 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:56.154968 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:56.156028 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:56.156260 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:56.156376 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:56.159621 (+ 3245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:56.159624 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:56.159624 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:56.159636 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:56.162336 (+ 2700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:56.163088 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:56.163090 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:56.163090 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:56.163176 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:56.163452 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:56.163453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:56.163454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:56.163727 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:56.163820 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:56.164179 (+ 359us) server_negotiation.cc:299] Negotiation successful
0619 17:02:56.164290 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 17:02:56.164408 11659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:56.153714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:42773 (local address 127.0.0.1:33812)
0619 17:02:56.153791 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:56.153804 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:56.153861 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:56.154044 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:56.154045 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:56.154242 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:56.154293 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:56.154295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:56.155058 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:56.155060 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:56.155891 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:56.155896 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:56.156379 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:56.156380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:56.156449 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:56.157085 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:02:56.157093 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:56.159439 (+ 2346us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:56.162546 (+ 3107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:56.162555 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:56.162556 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:56.162956 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:56.163280 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:56.163281 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:56.163281 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:56.163366 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:56.163819 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:56.163820 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:56.163900 (+ 80us) client_negotiation.cc:769] Sending connection context
0619 17:02:56.164124 (+ 224us) client_negotiation.cc:240] Negotiation successful
0619 17:02:56.164273 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260619 17:02:56.166388 11657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 035b5a44b0434ecb9eb79d53fd9d92c0: Bootstrap starting.
I20260619 17:02:56.167305 11657 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 035b5a44b0434ecb9eb79d53fd9d92c0: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:56.167661 11657 log.cc:824] T 00000000000000000000000000000000 P 035b5a44b0434ecb9eb79d53fd9d92c0: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:56.168757 11657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 035b5a44b0434ecb9eb79d53fd9d92c0: No bootstrap required, opened a new log
I20260619 17:02:56.171159 11657 raft_consensus.cc:348] T 00000000000000000000000000000000 P 035b5a44b0434ecb9eb79d53fd9d92c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e2f9d10fb448f5826a5906f54d8f51" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 38663 } } peers { permanent_uuid: "035b5a44b0434ecb9eb79d53fd9d92c0" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38811 } } peers { permanent_uuid: "e078cb3b5e104607a36d018f416ecc69" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 42773 } }
I20260619 17:02:56.171342 11657 raft_consensus.cc:729] T 00000000000000000000000000000000 P 035b5a44b0434ecb9eb79d53fd9d92c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 035b5a44b0434ecb9eb79d53fd9d92c0, State: Initialized, Role: FOLLOWER
I20260619 17:02:56.171430 11657 consensus_queue.cc:260] T 00000000000000000000000000000000 P 035b5a44b0434ecb9eb79d53fd9d92c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e2f9d10fb448f5826a5906f54d8f51" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 38663 } } peers { permanent_uuid: "035b5a44b0434ecb9eb79d53fd9d92c0" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38811 } } peers { permanent_uuid: "e078cb3b5e104607a36d018f416ecc69" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 42773 } }
I20260619 17:02:56.171912 11752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 035b5a44b0434ecb9eb79d53fd9d92c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e2f9d10fb448f5826a5906f54d8f51" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 38663 } } peers { permanent_uuid: "035b5a44b0434ecb9eb79d53fd9d92c0" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38811 } } peers { permanent_uuid: "e078cb3b5e104607a36d018f416ecc69" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 42773 } } }
I20260619 17:02:56.172003 11657 sys_catalog.cc:564] T 00000000000000000000000000000000 P 035b5a44b0434ecb9eb79d53fd9d92c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:02:56.172032 11752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 035b5a44b0434ecb9eb79d53fd9d92c0 [sys.catalog]: This master's current role is: FOLLOWER
W20260619 17:02:56.174321 11763 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 035b5a44b0434ecb9eb79d53fd9d92c0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 17:02:56.174453 11763 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 17:02:56.174291 11589 master_runner.cc:186] Error getting master registration for 127.20.231.190:38663: OK, instance_id { permanent_uuid: "b6e2f9d10fb448f5826a5906f54d8f51" instance_seqno: 1781888575731416 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 17:02:56.177484 11589 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 17:02:56.182617 11582 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.20.231.188" port: 42773 } attempt: 3
Jun 19 17:02:56 dist-test-slave-vnbj krb5kdc[11501](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781888575, etypes {rep=17 tkt=17 ses=17}, kudu/127.20.231.190@KRBTEST.COM for kudu/127.20.231.188@KRBTEST.COM
I20260619 17:02:56.193220 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:56.183034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.20.231.188:42773 (local address 127.0.0.1:33820)
0619 17:02:56.183101 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0619 17:02:56.183116 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 17:02:56.183164 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 17:02:56.183331 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 17:02:56.183332 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 17:02:56.183534 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0619 17:02:56.183584 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:56.183586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:56.184551 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:56.184552 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:56.185759 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 17:02:56.185764 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 17:02:56.186344 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 17:02:56.186346 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 17:02:56.186464 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:56.187617 (+ 1153us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 17:02:56.187624 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0619 17:02:56.189554 (+ 1930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 17:02:56.192004 (+ 2450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:56.192006 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:56.192006 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:56.192251 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:56.192499 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 17:02:56.192499 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 17:02:56.192500 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 17:02:56.192559 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 17:02:56.192857 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 17:02:56.192858 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 17:02:56.192909 (+ 51us) client_negotiation.cc:769] Sending connection context
0619 17:02:56.192974 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 17:02:56.193102 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 17:02:56.193674 11736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:56.183116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33820 (local address 127.20.231.188:42773)
0619 17:02:56.183159 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:56.183161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:56.183171 (+ 10us) server_negotiation.cc:407] Connection header received
0619 17:02:56.183248 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:56.183249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:56.183262 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:56.183314 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:56.183684 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:56.184442 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:56.185925 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:56.186197 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:56.186267 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:56.189695 (+ 3428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:56.189697 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:56.189698 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:56.189707 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:56.191869 (+ 2162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:56.192333 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:56.192334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:56.192335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:56.192425 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:56.192623 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:56.192624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:56.192624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:56.192773 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:56.192832 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:56.193481 (+ 649us) server_negotiation.cc:299] Negotiation successful
0619 17:02:56.193578 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260619 17:02:56.196290 11582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6e2f9d10fb448f5826a5906f54d8f51: Bootstrap starting.
I20260619 17:02:56.197194 11582 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6e2f9d10fb448f5826a5906f54d8f51: Neither blocks nor log segments found. Creating new log.
I20260619 17:02:56.197566 11582 log.cc:824] T 00000000000000000000000000000000 P b6e2f9d10fb448f5826a5906f54d8f51: Log is configured to *not* fsync() on all Append() calls
I20260619 17:02:56.198668 11582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6e2f9d10fb448f5826a5906f54d8f51: No bootstrap required, opened a new log
I20260619 17:02:56.200325 11582 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6e2f9d10fb448f5826a5906f54d8f51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e2f9d10fb448f5826a5906f54d8f51" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 38663 } } peers { permanent_uuid: "035b5a44b0434ecb9eb79d53fd9d92c0" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38811 } } peers { permanent_uuid: "e078cb3b5e104607a36d018f416ecc69" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 42773 } }
I20260619 17:02:56.200542 11582 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6e2f9d10fb448f5826a5906f54d8f51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6e2f9d10fb448f5826a5906f54d8f51, State: Initialized, Role: FOLLOWER
I20260619 17:02:56.200654 11582 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6e2f9d10fb448f5826a5906f54d8f51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e2f9d10fb448f5826a5906f54d8f51" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 38663 } } peers { permanent_uuid: "035b5a44b0434ecb9eb79d53fd9d92c0" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38811 } } peers { permanent_uuid: "e078cb3b5e104607a36d018f416ecc69" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 42773 } }
I20260619 17:02:56.201079 11556 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6e2f9d10fb448f5826a5906f54d8f51 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 17:02:56.201061 11582 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6e2f9d10fb448f5826a5906f54d8f51 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 17:02:56.201218 11556 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6e2f9d10fb448f5826a5906f54d8f51 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:56.202203 11556 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6e2f9d10fb448f5826a5906f54d8f51 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e2f9d10fb448f5826a5906f54d8f51" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 38663 } } peers { permanent_uuid: "035b5a44b0434ecb9eb79d53fd9d92c0" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38811 } } peers { permanent_uuid: "e078cb3b5e104607a36d018f416ecc69" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 42773 } }
I20260619 17:02:56.202564 11556 leader_election.cc:288] T 00000000000000000000000000000000 P b6e2f9d10fb448f5826a5906f54d8f51 [CANDIDATE]: Term 1 election: Requested vote from peers 035b5a44b0434ecb9eb79d53fd9d92c0 (127.20.231.189:38811), e078cb3b5e104607a36d018f416ecc69 (127.20.231.188:42773)
I20260619 17:02:56.203179 11706 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b6e2f9d10fb448f5826a5906f54d8f51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e078cb3b5e104607a36d018f416ecc69"
I20260619 17:02:56.203329 11706 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e078cb3b5e104607a36d018f416ecc69 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:56.204450 11706 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e078cb3b5e104607a36d018f416ecc69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6e2f9d10fb448f5826a5906f54d8f51 in term 1.
I20260619 17:02:56.204555 11631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b6e2f9d10fb448f5826a5906f54d8f51" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "035b5a44b0434ecb9eb79d53fd9d92c0"
I20260619 17:02:56.204701 11631 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 035b5a44b0434ecb9eb79d53fd9d92c0 [term 0 FOLLOWER]: Advancing to term 1
I20260619 17:02:56.204768 11530 leader_election.cc:302] T 00000000000000000000000000000000 P b6e2f9d10fb448f5826a5906f54d8f51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b6e2f9d10fb448f5826a5906f54d8f51, e078cb3b5e104607a36d018f416ecc69; no voters:
I20260619 17:02:56.205809 11631 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 035b5a44b0434ecb9eb79d53fd9d92c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6e2f9d10fb448f5826a5906f54d8f51 in term 1.
I20260619 17:02:56.206792 11766 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6e2f9d10fb448f5826a5906f54d8f51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e2f9d10fb448f5826a5906f54d8f51" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 38663 } } peers { permanent_uuid: "035b5a44b0434ecb9eb79d53fd9d92c0" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38811 } } peers { permanent_uuid: "e078cb3b5e104607a36d018f416ecc69" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 42773 } } }
I20260619 17:02:56.207005 11766 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6e2f9d10fb448f5826a5906f54d8f51 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 17:02:56.207089 11766 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6e2f9d10fb448f5826a5906f54d8f51 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 17:02:56.207171 11766 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6e2f9d10fb448f5826a5906f54d8f51 [term 1 LEADER]: Becoming Leader. State: Replica: b6e2f9d10fb448f5826a5906f54d8f51, State: Running, Role: LEADER
I20260619 17:02:56.207273 11766 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6e2f9d10fb448f5826a5906f54d8f51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e2f9d10fb448f5826a5906f54d8f51" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 38663 } } peers { permanent_uuid: "035b5a44b0434ecb9eb79d53fd9d92c0" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38811 } } peers { permanent_uuid: "e078cb3b5e104607a36d018f416ecc69" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 42773 } }
I20260619 17:02:56.208130 11766 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6e2f9d10fb448f5826a5906f54d8f51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6e2f9d10fb448f5826a5906f54d8f51. Latest consensus state: current_term: 1 leader_uuid: "b6e2f9d10fb448f5826a5906f54d8f51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e2f9d10fb448f5826a5906f54d8f51" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 38663 } } peers { permanent_uuid: "035b5a44b0434ecb9eb79d53fd9d92c0" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38811 } } peers { permanent_uuid: "e078cb3b5e104607a36d018f416ecc69" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 42773 } } }
I20260619 17:02:56.208249 11766 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6e2f9d10fb448f5826a5906f54d8f51 [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:56.209263 11514 master_runner.cc:263] No leader master found from master b6e2f9d10fb448f5826a5906f54d8f51
I20260619 17:02:56.209322 11514 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 17:02:56.210043 11784 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 17:02:56.210709 11784 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 17:02:56.218245 11585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:56.208692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56218 (local address 127.20.231.190:38663)
0619 17:02:56.208734 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:56.208737 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:56.209579 (+ 842us) server_negotiation.cc:407] Connection header received
0619 17:02:56.209684 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:56.209686 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:56.209697 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:56.209727 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:56.210585 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:56.211394 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:56.212585 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:56.212886 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:56.212929 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:56.214718 (+ 1789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:56.214720 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:56.214720 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:56.214728 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:56.216786 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:56.217238 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:56.217239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:56.217239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:56.217303 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:56.217521 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:56.217522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:56.217522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:56.217656 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:56.217734 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:56.217978 (+ 244us) server_negotiation.cc:299] Negotiation successful
0619 17:02:56.218090 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260619 17:02:56.227459 11736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:56.209225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33836 (local address 127.20.231.188:42773)
0619 17:02:56.209389 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:56.209391 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:56.211723 (+ 2332us) server_negotiation.cc:407] Connection header received
0619 17:02:56.211837 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:56.211838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:56.211850 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:56.211882 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:56.212760 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:56.213540 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:56.214623 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:56.215105 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:56.215155 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:56.224206 (+ 9051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:56.224210 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:56.224210 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:56.224219 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:56.225993 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:56.226512 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:56.226513 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:56.226514 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:56.226576 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:56.226776 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:56.226777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:56.226778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:56.226953 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:56.227029 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:56.227254 (+ 225us) server_negotiation.cc:299] Negotiation successful
0619 17:02:56.227348 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260619 17:02:56.228192 11706 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e078cb3b5e104607a36d018f416ecc69 [term 1 FOLLOWER]: Refusing update from remote peer b6e2f9d10fb448f5826a5906f54d8f51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 17:02:56.228590 11766 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b6e2f9d10fb448f5826a5906f54d8f51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e078cb3b5e104607a36d018f416ecc69" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 42773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 17:02:56.229148 11535 tablet.cc:2369] T 00000000000000000000000000000000 P b6e2f9d10fb448f5826a5906f54d8f51: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 17:02:56.229316 11661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:56.209219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39928 (local address 127.20.231.189:38811)
0619 17:02:56.209260 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:56.209262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:56.215628 (+ 6366us) server_negotiation.cc:407] Connection header received
0619 17:02:56.215748 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:56.215750 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:56.215763 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:56.215796 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:56.216562 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:56.217323 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:56.218478 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:56.218795 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:56.218840 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:56.220847 (+ 2007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 17:02:56.220849 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 17:02:56.220849 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 17:02:56.220858 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 17:02:56.222830 (+ 1972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:56.228315 (+ 5485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:56.228316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:56.228317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:56.228410 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 17:02:56.228643 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 17:02:56.228644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 17:02:56.228644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0619 17:02:56.228800 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 17:02:56.228899 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:56.229117 (+ 218us) server_negotiation.cc:299] Negotiation successful
0619 17:02:56.229217 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260619 17:02:56.230057 11738 sys_catalog.cc:455] T 00000000000000000000000000000000 P e078cb3b5e104607a36d018f416ecc69 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6e2f9d10fb448f5826a5906f54d8f51. Latest consensus state: current_term: 1 leader_uuid: "b6e2f9d10fb448f5826a5906f54d8f51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e2f9d10fb448f5826a5906f54d8f51" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 38663 } } peers { permanent_uuid: "035b5a44b0434ecb9eb79d53fd9d92c0" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38811 } } peers { permanent_uuid: "e078cb3b5e104607a36d018f416ecc69" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 42773 } } }
I20260619 17:02:56.230149 11738 sys_catalog.cc:458] T 00000000000000000000000000000000 P e078cb3b5e104607a36d018f416ecc69 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 17:02:56.230976 11631 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 035b5a44b0434ecb9eb79d53fd9d92c0 [term 1 FOLLOWER]: Refusing update from remote peer b6e2f9d10fb448f5826a5906f54d8f51: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 17:02:56.231271 11766 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b6e2f9d10fb448f5826a5906f54d8f51 [LEADER]: Connected to new peer: Peer: permanent_uuid: "035b5a44b0434ecb9eb79d53fd9d92c0" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 17:02:56.232681 11752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 035b5a44b0434ecb9eb79d53fd9d92c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6e2f9d10fb448f5826a5906f54d8f51. Latest consensus state: current_term: 1 leader_uuid: "b6e2f9d10fb448f5826a5906f54d8f51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e2f9d10fb448f5826a5906f54d8f51" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 38663 } } peers { permanent_uuid: "035b5a44b0434ecb9eb79d53fd9d92c0" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38811 } } peers { permanent_uuid: "e078cb3b5e104607a36d018f416ecc69" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 42773 } } }
I20260619 17:02:56.232789 11752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 035b5a44b0434ecb9eb79d53fd9d92c0 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 17:02:56.234012 11766 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6e2f9d10fb448f5826a5906f54d8f51 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b6e2f9d10fb448f5826a5906f54d8f51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e2f9d10fb448f5826a5906f54d8f51" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 38663 } } peers { permanent_uuid: "035b5a44b0434ecb9eb79d53fd9d92c0" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38811 } } peers { permanent_uuid: "e078cb3b5e104607a36d018f416ecc69" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 42773 } } }
I20260619 17:02:56.234130 11766 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6e2f9d10fb448f5826a5906f54d8f51 [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:56.234192 11738 sys_catalog.cc:455] T 00000000000000000000000000000000 P e078cb3b5e104607a36d018f416ecc69 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b6e2f9d10fb448f5826a5906f54d8f51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e2f9d10fb448f5826a5906f54d8f51" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 38663 } } peers { permanent_uuid: "035b5a44b0434ecb9eb79d53fd9d92c0" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38811 } } peers { permanent_uuid: "e078cb3b5e104607a36d018f416ecc69" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 42773 } } }
I20260619 17:02:56.234287 11738 sys_catalog.cc:458] T 00000000000000000000000000000000 P e078cb3b5e104607a36d018f416ecc69 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 17:02:56.234783 11789 mvcc.cc:205] Tried to move back new op lower bound from 7298615608226848768 to 7298615608147439616. Current Snapshot: MvccSnapshot[applied={T|T < 7298615608226848768}]
I20260619 17:02:56.234944 11790 mvcc.cc:205] Tried to move back new op lower bound from 7298615608226848768 to 7298615608147439616. Current Snapshot: MvccSnapshot[applied={T|T < 7298615608226848768}]
I20260619 17:02:56.235837 11766 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6e2f9d10fb448f5826a5906f54d8f51 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b6e2f9d10fb448f5826a5906f54d8f51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e2f9d10fb448f5826a5906f54d8f51" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 38663 } } peers { permanent_uuid: "035b5a44b0434ecb9eb79d53fd9d92c0" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38811 } } peers { permanent_uuid: "e078cb3b5e104607a36d018f416ecc69" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 42773 } } }
I20260619 17:02:56.235948 11766 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6e2f9d10fb448f5826a5906f54d8f51 [sys.catalog]: This master's current role is: LEADER
I20260619 17:02:56.236514 11752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 035b5a44b0434ecb9eb79d53fd9d92c0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b6e2f9d10fb448f5826a5906f54d8f51" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b6e2f9d10fb448f5826a5906f54d8f51" member_type: VOTER last_known_addr { host: "127.20.231.190" port: 38663 } } peers { permanent_uuid: "035b5a44b0434ecb9eb79d53fd9d92c0" member_type: VOTER last_known_addr { host: "127.20.231.189" port: 38811 } } peers { permanent_uuid: "e078cb3b5e104607a36d018f416ecc69" member_type: VOTER last_known_addr { host: "127.20.231.188" port: 42773 } } }
I20260619 17:02:56.236599 11752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 035b5a44b0434ecb9eb79d53fd9d92c0 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 17:02:56.236892 11792 mvcc.cc:205] Tried to move back new op lower bound from 7298615608226848768 to 7298615608147439616. Current Snapshot: MvccSnapshot[applied={T|T < 7298615608226848768}]
I20260619 17:02:56.243522 11784 catalog_manager.cc:1347] Generated new cluster ID: 0cbdd159b7134caa9672be42a210b93e
I20260619 17:02:56.243618 11784 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 17:02:56.263746 11784 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 17:02:56.265811 11784 catalog_manager.cc:1504] Loading token signing keys...
I20260619 17:02:56.278771 11784 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6e2f9d10fb448f5826a5906f54d8f51: Generated new TSK 0
I20260619 17:02:56.279094 11784 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 17:02:56.308772 11661 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 17:02:56.307581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39930 (local address 127.20.231.189:38811)
0619 17:02:56.307632 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:56.307634 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:56.307978 (+ 344us) server_negotiation.cc:407] Connection header received
0619 17:02:56.308094 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:56.308095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:56.308107 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:56.308140 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:56.308699 (+ 559us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39930: BlockingRecv error: recv got EOF from 127.0.0.1:39930 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260619 17:02:56.310571 11585 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 17:02:56.309337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56222 (local address 127.20.231.190:38663)
0619 17:02:56.309415 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:56.309417 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:56.309799 (+ 382us) server_negotiation.cc:407] Connection header received
0619 17:02:56.310010 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:56.310011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:56.310025 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:56.310061 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:56.310489 (+ 428us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56222: BlockingRecv error: recv got EOF from 127.0.0.1:56222 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260619 17:02:56.311640 11736 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 17:02:56.309661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33838 (local address 127.20.231.188:42773)
0619 17:02:56.309705 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:56.309707 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:56.311030 (+ 1323us) server_negotiation.cc:407] Connection header received
0619 17:02:56.311141 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:56.311143 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:56.311156 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:56.311189 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:56.311573 (+ 384us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33838: BlockingRecv error: recv got EOF from 127.0.0.1:33838 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260619 17:02:56.314369 11661 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 17:02:56.313470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39940 (local address 127.20.231.189:38811)
0619 17:02:56.313518 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:56.313521 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:56.313718 (+ 197us) server_negotiation.cc:407] Connection header received
0619 17:02:56.313834 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:56.313835 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:56.313872 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:56.313909 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:56.314306 (+ 397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39940: BlockingRecv error: recv got EOF from 127.0.0.1:39940 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260619 17:02:56.317988 11661 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 17:02:56.316958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39952 (local address 127.20.231.189:38811)
0619 17:02:56.317006 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:56.317008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:56.317313 (+ 305us) server_negotiation.cc:407] Connection header received
0619 17:02:56.317436 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:56.317438 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:56.317451 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:56.317486 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:56.317924 (+ 438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39952: BlockingRecv error: recv got EOF from 127.0.0.1:39952 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260619 17:02:56.322506 11661 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 17:02:56.321375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39962 (local address 127.20.231.189:38811)
0619 17:02:56.321429 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:56.321432 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:56.321781 (+ 349us) server_negotiation.cc:407] Connection header received
0619 17:02:56.321930 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:56.321931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:56.321946 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:56.321994 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:56.322440 (+ 446us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39962: BlockingRecv error: recv got EOF from 127.0.0.1:39962 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260619 17:02:56.328992 11661 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 17:02:56.327978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39968 (local address 127.20.231.189:38811)
0619 17:02:56.328032 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:56.328035 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:56.328350 (+ 315us) server_negotiation.cc:407] Connection header received
0619 17:02:56.328461 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:56.328462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:56.328474 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:56.328506 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:56.328926 (+ 420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39968: BlockingRecv error: recv got EOF from 127.0.0.1:39968 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260619 17:02:56.339772 11661 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 17:02:56.338443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39976 (local address 127.20.231.189:38811)
0619 17:02:56.338495 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:56.338498 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:56.339080 (+ 582us) server_negotiation.cc:407] Connection header received
0619 17:02:56.339209 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:56.339211 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:56.339223 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:56.339256 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:56.339694 (+ 438us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39976: BlockingRecv error: recv got EOF from 127.0.0.1:39976 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260619 17:02:56.358103 11661 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 17:02:56.357252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39978 (local address 127.20.231.189:38811)
0619 17:02:56.357300 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:56.357302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:56.357433 (+ 131us) server_negotiation.cc:407] Connection header received
0619 17:02:56.357538 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:56.357539 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:56.357552 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:56.357618 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:56.358041 (+ 423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39978: BlockingRecv error: recv got EOF from 127.0.0.1:39978 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260619 17:02:56.392493 11661 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 17:02:56.391731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39980 (local address 127.20.231.189:38811)
0619 17:02:56.391793 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:56.391796 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:56.391934 (+ 138us) server_negotiation.cc:407] Connection header received
0619 17:02:56.392026 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:56.392027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:56.392039 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:56.392091 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:56.392435 (+ 344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39980: BlockingRecv error: recv got EOF from 127.0.0.1:39980 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
W20260619 17:02:56.458593 11661 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 17:02:56.457792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39982 (local address 127.20.231.189:38811)
0619 17:02:56.457891 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:56.457899 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:56.457912 (+ 13us) server_negotiation.cc:407] Connection header received
0619 17:02:56.457946 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:56.457947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:56.457958 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:56.458025 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:56.458507 (+ 482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39982: BlockingRecv error: recv got EOF from 127.0.0.1:39982 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260619 17:02:56.588871 11661 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 17:02:56.588022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39990 (local address 127.20.231.189:38811)
0619 17:02:56.588183 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:56.588185 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:56.588196 (+ 11us) server_negotiation.cc:407] Connection header received
0619 17:02:56.588224 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:56.588225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:56.588237 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:56.588303 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:56.588792 (+ 489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39990: BlockingRecv error: recv got EOF from 127.0.0.1:39990 (error 108)
Metrics: {"server-negotiator.queue_time_us":127}
W20260619 17:02:56.847175 11661 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 17:02:56.846342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39992 (local address 127.20.231.189:38811)
0619 17:02:56.846426 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:56.846429 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:56.846443 (+ 14us) server_negotiation.cc:407] Connection header received
0619 17:02:56.846507 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:56.846508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:56.846519 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:56.846572 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 17:02:56.847095 (+ 523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39992: BlockingRecv error: recv got EOF from 127.0.0.1:39992 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 17:02:57.134691 11749 catalog_manager.cc:1259] Loaded cluster ID: 0cbdd159b7134caa9672be42a210b93e
I20260619 17:02:57.134785 11749 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e078cb3b5e104607a36d018f416ecc69: loading cluster ID for follower catalog manager: success
I20260619 17:02:57.135731 11749 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e078cb3b5e104607a36d018f416ecc69: acquiring CA information for follower catalog manager: success
I20260619 17:02:57.136143 11749 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e078cb3b5e104607a36d018f416ecc69: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 17:02:57.175143 11763 catalog_manager.cc:1259] Loaded cluster ID: 0cbdd159b7134caa9672be42a210b93e
I20260619 17:02:57.175246 11763 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 035b5a44b0434ecb9eb79d53fd9d92c0: loading cluster ID for follower catalog manager: success
I20260619 17:02:57.176122 11763 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 035b5a44b0434ecb9eb79d53fd9d92c0: acquiring CA information for follower catalog manager: success
I20260619 17:02:57.176488 11763 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 035b5a44b0434ecb9eb79d53fd9d92c0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 17:02:57.363701 11861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:57.360522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60572 (local address 127.20.231.189:38811)
0619 17:02:57.360685 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:57.360687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:57.360701 (+ 14us) server_negotiation.cc:407] Connection header received
0619 17:02:57.360797 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:57.360798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:57.360884 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:57.361019 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 17:02:57.361400 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:57.361936 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:57.362771 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:57.363048 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:57.363139 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:57.363332 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 17:02:57.363451 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 17:02:57.363545 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:57.363568 (+ 23us) server_negotiation.cc:299] Negotiation successful
0619 17:02:57.363592 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":60,"threads_started":1}
I20260619 17:02:57.368743 11867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:57.365926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41030 (local address 127.20.231.188:42773)
0619 17:02:57.366059 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:57.366061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:57.366069 (+ 8us) server_negotiation.cc:407] Connection header received
0619 17:02:57.366110 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:57.366110 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:57.366148 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:57.366227 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 17:02:57.366463 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:57.367030 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:57.367884 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:57.368165 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:57.368227 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:57.368443 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 17:02:57.368552 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 17:02:57.368609 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:57.368654 (+ 45us) server_negotiation.cc:299] Negotiation successful
0619 17:02:57.368673 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260619 17:02:57.374101 11861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:57.370817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60576 (local address 127.20.231.189:38811)
0619 17:02:57.370886 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:57.370888 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:57.371063 (+ 175us) server_negotiation.cc:407] Connection header received
0619 17:02:57.371181 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:57.371182 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:57.371194 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:57.371238 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 17:02:57.371401 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:57.372210 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:57.373216 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:57.373535 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:57.373624 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:57.373788 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 17:02:57.373822 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 17:02:57.373849 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:57.373951 (+ 102us) server_negotiation.cc:299] Negotiation successful
0619 17:02:57.374012 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 17:02:57.374557 11873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:57.370793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60278 (local address 127.20.231.190:38663)
0619 17:02:57.370937 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:57.370939 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:57.371040 (+ 101us) server_negotiation.cc:407] Connection header received
0619 17:02:57.371167 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:57.371168 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:57.371233 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:57.371351 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 17:02:57.371467 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:57.372127 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:57.373355 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:57.373687 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:57.373754 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:57.373947 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 17:02:57.374076 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 17:02:57.374137 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:57.374475 (+ 338us) server_negotiation.cc:299] Negotiation successful
0619 17:02:57.374499 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":80,"threads_started":1}
I20260619 17:02:57.374699 11867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 17:02:57.371094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41042 (local address 127.20.231.188:42773)
0619 17:02:57.371140 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0619 17:02:57.371142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0619 17:02:57.371356 (+ 214us) server_negotiation.cc:407] Connection header received
0619 17:02:57.371547 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 17:02:57.371548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 17:02:57.371559 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 17:02:57.371616 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 17:02:57.371838 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:57.372640 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:57.373682 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 17:02:57.373984 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 17:02:57.374047 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 17:02:57.374349 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 17:02:57.374380 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 17:02:57.374409 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0619 17:02:57.374593 (+ 184us) server_negotiation.cc:299] Negotiation successful
0619 17:02:57.374617 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260619 17:02:57.376717 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 11514
I20260619 17:02:57.383006 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 11589
I20260619 17:02:57.388978 21406 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGfxJfS/build/release/bin/kudu with pid 11664
2026-06-19T17:02:57Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3000 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5982 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (276052 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST